acceptodds
Withdrawn from ICLR 2027

ViB-OPD: Visual Evidence Mining and Boundary-Balanced Video Distillation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing approaches commonly use supervised fine-tuning (SFT) or reinforcement learning (RL) to develop video understanding and reasoning; however, SFT is constrained by the quality and coverage of fixed demonstrations, while RL often relies on sparse final-answer rewards that provide little direct guidance for intermediate reasoning. On-policy distillation (OPD) offers an alternative by providing teacher distributional supervision along trajectories generated by the current student. Yet applying OPD to video reasoning leaves two issues unresolved: teacher distributions can be limited by incomplete visual evidence, and token-wise aggregation can make supervision allocation depend on expression length. We propose ViB-OPD, comprising Visual Evidence Mining (VEM) and Boundary-Balanced OPD (BB-OPD). A stronger frozen teacher uses supplementary training-time visual context to guide the student without expanding its inference input. VEM combines global observation, question-conditioned localization, and optional local reinspection to organize candidate evidence into a separate contact sheet, enriching the teacher's visual basis. BB-OPD initializes semantic groups and combines local boundary perturbation with within-segment averaging and equal segment weights to reduce the influence of expression length and a single segmentation on supervision allocation. It preserves the original student trajectory and token-level distillation objective. Evaluation on six public benchmarks and across model families demonstrates training gains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodel, opd, video understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_Ld8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ViB-OPD trains a video-language student with feedback from a stronger teacher on the student's own responses. The authors add two steps. Visual Evidence Mining (VEM) gives the teacher a sheet of 16 frames chosen for the question, in addition to the student's 32 frames. Boundary-Balanced OPD (BB-OPD) divides the student's reasoning into meaningful segments and slightly shifts the boundaries. The loss averages the token-level reverse KL within each segment, then gives each segment equal weight. At inference, the student uses its usual input.

Critical strengths
  1. The authors check some obvious alternative explanations. In Figure 2, the teacher gets 32 frames plus a sheet of 16 frames in every setting. VEM scores 0.93 and 0.89 points above the uniform and random alternatives. Table 3 also compares different ways to group tokens, including random groups with matched lengths. I found these comparisons useful: they test whether choosing the frames and segments matters.

  2. With 32 frames, both Qwen3-VL-8B and Qwen3.5-9B improve on all six benchmarks. The authors report using the same hyperparameters for the second model. The extra work happens during training, so the student needs no extra frames or modules at inference. That makes the gains useful even if the training is more expensive.

Critical weaknesses
  1. I could not find the sources of the 1,000 training examples, how they were selected, or checks for shared videos with the benchmarks. Figure 3 compares settings on Video-MME without naming a separate development set. These details matter for assessing generalization.

  2. Segment weighting adds one point on Video-MME (63.52% vs. 62.52%), mostly from shifting boundaries. I would like a paired comparison that accounts for questions from the same video, plus the same controls on other benchmarks. The authors do not directly test whether teacher errors decrease, and they do not report the extra cost of the evidence builder and 32B model.

Questions for authors
  1. Where do the 1,000 training examples come from, and how did you choose them? Did you check for shared videos with each benchmark? I also could not tell which data you used to choose the checkpoint, the 10% boundary shift, and K=128.

  2. Do the gains over VEM+Token and VEM+Semantic hold in a paired comparison that groups questions by video? I would be interested in the same comparisons on the other five benchmarks, since most of the evidence for the weighting comes from Video-MME.

  3. Which tokens does the process mask keep? How do you shift the segment boundaries, and what happens when the model fails to divide the reasoning correctly? How much extra compute do the evidence builder and the 32B model need compared with ordinary OPD?

Reviewer_W5h8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed ViB-OPD to improve on-policy distillation for video reasoning. The motivation is that existing methods may provide inaccurate supervision from the teacher, so this paper proposed Visual Evidence Mining (VEM) to help the teacher identify relevant video content, and Boundary-Balanced OPD (BB-OPD) to balance the training loss across reasoning steps. The approach only changes the training process and does not add inference overhead, and the improvements were shown on 6 video understanding benchmarks across different models.

Critical strengths
  1. The motivation for improving teacher supervision with better visual evidence and more balanced reasoning steps is intuitive and convincing.
  2. I appreciate the experiments on 6 video understanding benchmarks across 2 model families to show the improvements.
  3. I also appreciate the ablation studies. E.g., the comparisons with random and uniform frame sampling show the effectiveness of VEM, while the comparisons with different segmentation strategies support the benefit of BB-OPD.
  4. Good inference efficiency (no inference overhead).
Critical weaknesses
  1. On-policy distillation is used in On-Policy Distillation of Language Models (ICLR 2024), and knowledge distillation and loss reweighting are well-established techniques. Thus, the technical contribution is weak.
  2. Several small improvements, e.g., <+1% on VideoMMMU, VSI, and TempCompass on Qwen3-VL-8B (32 frames).
  3. The experiments only cover student models up to 9B parameters. It is unclear whether the method would still be effective for much larger models, e.g., 100B+, which already has a strong performance.
  4. Table 3(a) shows that BB-OPD alone decreases performance from 62.07% to 61.89%. Although the authors provide an explanation, its effectiveness without VEM is questionable.
  5. The method requires large teacher models which slow down the training and increase memory usage, and this method also further introduces costs, e.g., VEM's visual evidence generation and BB-OPD's semantic segmentation.
Questions for authors
  1. Would this method work with a 100B+ model, e.g., Qwen3.5-122B-A10B (weakness 3)?
  2. What is the training speed overhead (weakness 5)?
Reviewer_wkdZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper applies on-policy distillation to video reasoning with two components. Visual Evidence Mining (VEM) builds a 16-frame contact sheet of question-relevant moments, and only the teacher sees it during training. Boundary-Balanced OPD (BB-OPD) splits each student rollout into semantic units, slightly perturbs the unit boundaries, and weights all units equally. At inference the student uses only its original 32-frame input. Across six benchmarks and two model families, the method gives consistent but modest gains. For the 8B student, Video-MME rises from 61.70% to 63.52%, 1.00 point above token-averaged OPD with the same teacher. However, most effects in the ablations are below one point, come from a single run, and are measured on the same benchmark used to tune the method.

Critical strengths

1.Giving only the teacher question-relevant evidence is a sensible use of privileged information, and it adds no inference cost. Figure 2 includes the key control: with the teacher’s budget fixed at 32 + 16 frames, the VEM sheet (63.52%) beats uniform (62.59%) and random (62.63%) sheets.

2.The gains replicate across two model families and two frame budgets without re-tuning (Table 1). The 9B student gains 1.82 and 1.37 points on average, so the recipe is not tied to one backbone.

Critical weaknesses

1.My main concern is that the gains are small and the evidence behind them is thin. All design choices, including the 10% perturbation scale and K = 128, are tuned on Video-MME, which is also the main reported result. The improvement over token-averaged OPD is 1.00 point, roughly 27 questions out of 2,700, and every number comes from a single short run with no seeds or significance tests. At this scale I am not convinced the differences in Table 3 and Figure 3 are larger than run-to-run noise. The ablations are also only done on Video-MME with Qwen3-VL, so the cross-family results support the overall recipe but not the specific claims about BB-OPD.

2.I am not convinced that BB-OPD works as claimed. In Table 3a, BB-OPD alone (61.89%) is actually worse than plain token averaging (62.07%), which undercuts the paper’s motivation that length-based weighting is a problem. The explanation that it “amplifies noise” without good visual evidence sounds reasonable but is not tested. I also find it hard to see how the boundary perturbation can matter this much. Each rollout gets one random partition, and a 10% shift only moves a token or two between neighboring units. Yet it is credited with 0.82 points over fixed semantic grouping. Without some analysis of how the weights actually change, this could just be noise or a mild regularization effect.

Other concerns:

3.Some natural baselines are missing, such as the teacher’s own accuracy with and without the contact sheet, and SFT or GRPO on the same 1,000 examples. The Table 1 baselines use different backbones and protocols, so they do not say much about the method itself.

4.It is not clear where the training videos come from. The short/medium/long split looks like Video-MME’s, so the authors should explain how they avoided overlap with the test sets.

5.The “Global” bar in Figure 2 (62.26%) does not match either Global row in Table 3a, and the figure does not say which loss it uses.

6.Training cost is not reported, even though the pipeline runs a 32B model three times (evidence building, teacher scoring, and segmentation). It would help to compare against simply giving the student more frames at inference.

7.The examples in Figure 4 are picked by outcome and do not show that the VEM evidence is what fixed the answers.

8.Minor: “multimodel” in the keywords, a missing space in “subtitles.Our”, and no limitations section.

Questions for authors

1.Can the authors report mean and standard deviation over at least three training seeds for the base model, VEM + Token, VEM + Semantic (fixed), and ViB-OPD? Do the 1.00-point and 0.82-point gaps remain clearly above run-to-run variance.

2.Can the authors repeat the aggregation comparison (Table 3b) and the component ablation (Table 3a) on at least one benchmark other than Video-MME, and preferably on the Qwen3.5 family? Was any configuration choice made on data other than the Video-MME test set?

3.Why does BB-OPD hurt with the global-only teacher (61.89% vs. 62.07%)? Is there direct evidence for the “amplified noise” explanation, for example teacher entropy or teacher accuracy per segment type under the two teachers?

4.How large is the per-token weight change that the 10% perturbation actually induces, given the distribution of unit lengths? How does a control that adds random noise of the same magnitude to the fixed semantic weights compare with BB-OPD?

5.What are the accuracies of the 32B teacher on Video-MME with and without the contact sheet? How does ViB-OPD compare with SFT or RL baselines trained on the same 1,000 records with the same teacher?

Reviewer_brt4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ViB-OPD for on-policy distillation in video-language reasoning. The method has two components. Visual Evidence Mining (VEM) lets a frozen teacher use a training-only 16-frame contact sheet selected through global observation, question-conditioned temporal localization, and optional local reinspection, while the deployed student retains its original visual input. Boundary-Balanced OPD (BB-OPD) partitions the student's reasoning trajectory into semantic units, locally perturbs their boundaries, averages token-level reverse-KL losses within each unit, and gives all units equal weight. Experiments use 1,000 training records and evaluate two teacher–student model families on six public video benchmarks.

Critical strengths
  1. VEM is supported by a meaningful matched-budget control. With the teacher's 32-frame global input and 16-frame contact-sheet format fixed, uniform, random, and VEM-selected sheets obtain 62.59%, 62.63%, and 63.52% on Video-MME, respectively. This comparison provides evidence that question-conditioned evidence selection contributes beyond additional teacher-side frames. The extra visual processing remains training-only and does not increase the student's deployment input.

  2. The evaluation is broad and includes a useful cross-family replication. The method is evaluated on six public benchmarks and improves both Qwen3-VL-8B and Qwen3.5-9B configurations at the reported frame budgets. The second model family uses the same main hyperparameters, which reduces the concern that the entire result is specific to one backbone.

Critical weaknesses
  1. The provenance of the 1,000 training records is insufficiently specified. Section 4.1 reports their number and duration distribution, but does not identify the source videos, explain how the questions were obtained, or describe the separation protocol with respect to all six evaluation benchmarks. Although the paper states that training and evaluation indices are constructed separately and that no Video-MME training examples are used, the available description is insufficient to determine whether source videos or derived questions overlap with the other evaluation sets. This information is necessary to assess the validity and reproducibility of the reported cross-benchmark gains.

  2. The BB-OPD ablation does not isolate semantic boundary robustness from generic stochastic reweighting. Table 3b compares fixed semantic grouping and fixed length-matched random grouping against BB-OPD, but only BB-OPD combines its initialization with local boundary perturbation. The key matched control—applying the same perturbation rule to a non-semantic or length-matched random initialization—is absent. Consequently, the 0.82-point gain of BB-OPD over fixed semantic grouping could arise from generic stochastic changes in token weights rather than robustness to uncertainty around meaningful semantic boundaries. The fact that BB-OPD without VEM decreases Overall accuracy from 62.07% to 61.89% further shows that the proposed aggregation is not independently beneficial. The current evidence supports the complete recipe, but not the claimed explanation of why its aggregation component works.

Questions for authors
  1. What is the exact source of the 1,000 video–question training records, how were the questions obtained, and what deduplication or overlap checks were performed against the source videos, questions, and derived examples in all six evaluation benchmarks?

  2. What split was used to choose the aggregation method, boundary-perturbation scale, and teacher top-KK? Were 10% and K=128K=128 fixed using an independent validation set, or selected after inspecting the Video-MME test accuracies reported in Tables 2–3 and Figure 3?

  3. What happens when the same local perturbation rule used by BB-OPD is applied to a length-matched random or punctuation-based initialization? This matched comparison is needed to determine whether the gain depends on semantic boundary neighborhoods or follows from stochastic token reweighting more generally.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.