- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes LENS, a video self-distillation framework that considers teacher reliability and student evidence visibility. A frozen teacher receives sparse global frames augmented with dense frames from annotated evidence intervals. Teacher correct samples receive reverse-KL distillation, while teacher wrong/student correct samples receive pairwise supervision. Training data are filtered by the student’s coverage of the evidence interval. Experiments on four videoQA benchmarks show improvements, particularly on the 8B backbone, with ablations examining routing, visibility filtering, and divergence objectives.
Critical strengths
-
Useful analysis of privileged supervision. The paper distinguishes teacher accuracy from teaching utility and examines cases where additional visual evidence misleads the teacher.
-
Practical handling of teacher failures. Pairing correct student responses with incorrect teacher responses avoids blindly imitating the teacher and contributes a 1.88 point gain in the 8B ablation, without adding inference-time requirements.
Critical weaknesses
-
Questionable practical value of OPSD. In my own experiments, OPSD has not yielded useful improvements, making me skeptical of its practical value in this setting.
-
Missing mainstream benchmark evaluation. The selected benchmarks omit widely used evaluations such as VideoMME, VideoMMMU, and MVBench. This makes it difficult to assess the baseline results and reported gains.
-
Limited evidence for the learnability specific components. Visibility filtering contributes only 0.40 points, and reverse KL exceeds JSD by only 0.07 points. Gains over the strongest baseline are also small on 4B and 2B (0.57 and 0.34 points).
-
Insufficient comparison with closely related methods like Video-OPSD.
Questions for authors
-
Do the improvements persist on VideoMME, VideoMMMU, and MVBench under matched frame budgets and standard evaluation protocols?
-
What motivates the use of OPSD?
4 comments
Response to Reviewer P19i (Part 2/2)
W2&Q1: Missing mainstream benchmark evaluation. The selected benchmarks omit widely used evaluations such as VideoMME, VideoMMMU, and MVBench. Do the improvements persist on VideoMME, VideoMMMU, and MVBench under matched frame budgets and standard evaluation protocols?
Our four primary benchmarks were chosen because they provide GT intervals [t_start, t_end] to measure coverage and starvation regimes (§4.4, Table 3). Within our compute budget, we directly evaluated our trained Qwen3-VL-8B LENS checkpoint from Table 1 (without retraining) against Base under matched frame budgets on Video-MME (w/o subs, 64f) and LSDBench [8] (32f), an hour-long video benchmark targeting short actions under sparse sampling:
| Model (Qwen3-VL-8B) | Video-MME (64f) | LSDBench (32f) |
|---|---|---|
| Base | 61.7 | 55.3 |
| LENS (Ours) | 62.9 (+1.2) | 57.6 (+2.3) |
LENS improves over Base on both benchmarks (+1.2 on Video-MME, +2.3 on LSDBench) without retraining, confirming generalization to general and sparse-sampling long-video understanding. We will include these results in the revision.
W3: Limited evidence for the learnability specific components. Visibility filtering contributes only 0.40 points, and reverse KL exceeds JSD by only 0.07 points. Gains over the strongest baseline are also small on 4B and 2B (0.57 and 0.34 points).
Examined against their intended baselines, each result supports learnability and clear scaling behavior:
- Visibility goes beyond the final
+0.40step. On the teacher side (Table 8), keeping student-visible global frames (49.89) beats GT-only frames (Grounded,48.81) by +1.08 (+2.20 on QA-Ego4D, +1.71 on S-EMBER). On the data side, gains follow a clear starvation gradient (+4.23to+2.15, Figure 1d). Applied last in Table 2(a) after Reverse KL and routing (+1.88), visibility filtering still adds +1.30 on NExT-GQA (u16) and +0.61 on S-EMBER, diluted to+0.40only by unweighted 5-column averaging yet remaining highly significant (95% CI [+0.28, +0.52], ; LENS vs.OPSD-Answer:+2.19,95% CI [+1.67, +2.71], ). - Reverse KL vs. Forward KL (and why JSD agrees). Standard OPSD uses mode-covering Forward KL (
49.25), which penalizes unseen dense-frame modes and drops CG-Bench −0.93 below Base (31.96vs.32.89), whereas Reverse KL (49.89) gains +2.45 on QA-Ego4D and +1.19 on CG-Bench. JSD (,49.82) is a bounded divergence incorporating student-weighted KL and removing Forward KL’s infinite mode-covering penalty; both beating Forward KL (+0.64and+0.57) reinforces avoiding mode-covering over invisible evidence. - Why gains narrow on 4B and 2B. LENS ranks best at all scales (
+0.74on 2B,+3.88on 4B,+4.90on 8B over Base), while smaller margins stem from: (i) fewerT✗S✓samples on weaker backbones (as smaller students solve fewer sparse cases where the teacher errs), reducing self-reinforcement signal; and (ii) 2B instruction-following limits (Appendix C.1), requiring a letter-only prompt that bypasses early CoT tokens where 71.5% of repairs occur (Figure 4).
W4: Insufficient comparison with closely related methods like Video-OPSD.
Closely related video OPSD formulations, including concurrent Video-OPSD [9] and Clue-OPSD [10], are already evaluated in our paper (Table 1 & Table 8):
| Method / Configuration (8B) | Privileged Information | Avg. |
|---|---|---|
Self-Distilled Reasoner [5] (OPSD-Answer, Table 1) | GT answer | 49.98 |
Video-OPSD [9] / Clue-OPSD [10] (Grounded, Table 8) | Dense GT frames only | 48.81 |
| Contextualized Grounded Teacher (Ours, Table 8) | Sparse + dense GT (T✓) | 49.89 |
| LENS (Full Framework, Table 1) | Sparse + dense GT + routing | 52.17 |
On matched T✓ distillation (Table 8), adding global sparse context (49.89) beats the GT-only Grounded teacher of Video-OPSD [9] and Clue-OPSD [10] (48.81) by +1.08. On all data, routing T✗S✓ failures lifts LENS to 52.17 (+3.36 over Grounded, +2.19 over OPSD-Answer [5]). We will explicitly discuss these comparisons in §5.
Final Acknowledgment
We hope our responses resolve your concerns and are happy to clarify further. Thank you again for your thoughtful review.
References
[1] Kaur et al., Rethinking OPSD for Thinking Models, 2026. [2] Zhang et al., What Does Privileged Information Add to OPSD?, 2026. [3] Tian et al., What Should a Self-Teacher See?, 2026. [4] Agarwal et al., On-Policy Distillation of LMs, ICLR 2024. [5] Zhao et al., Self-Distilled Reasoner, 2026. [6] Feng et al., Video-R1, NeurIPS 2025. [7] Sun et al., video-SALMONN-o1, ICML 2025. [8] Qu et al., LSDBench, ICCV 2025. [9] Wang et al., Video-OPSD, 2026. [10] Wang et al., Clue-OPSD, 2026.
Thanks for the response. However, mainstream benchmarks are still missing, and VideoMME alone is insufficient to demonstrate the effectiveness of the method. Therefore, I will maintain my score.
Additional Results on Mainstream Benchmarks
We thank the reviewer for the prompt feedback. Over the past few days, we have completed the remaining evaluations on MVBench and VideoMMMU. Below, we report the unified results across all four additional benchmarks.
| Model (Qwen3-VL-8B) | Video-MME (64f) | MVBench (64f) | VideoMMMU (64f) | LSDBench (32f) |
|---|---|---|---|---|
| Base | 61.7 | 62.4 | 62.3 | 55.3 |
| LENS (Ours) | 62.9 (+1.2) | 63.8 (+1.4) | 62.8 (+0.5) | 57.6 (+2.3) |
LENS consistently improves over the Base model across all four benchmarks. Specifically, gains are most pronounced on LSDBench (+2.3), MVBench (+1.4), and Video-MME (+1.2), which rely on fine-grained temporal action perception and long-video evidence localization under sparse sampling, directly matching LENS’s design. Meanwhile, VideoMMMU focuses on lecture-style STEM disciplinary knowledge rather than temporal event grounding (with zero STEM data in our training set), so its gain is naturally smaller (+0.5) while still showing positive out-of-domain generalization without forgetting.
We will include these results in the revised paper, and hope they address your remaining concern.
Response to Reviewer P19i (Part 1/2)
Initial Acknowledgment
We thank the reviewer for the constructive feedback and appreciate the recognition of our analysis distinguishing teacher accuracy from teaching utility, as well as the +1.88 gain from handling teacher failures without inference overhead. Below, we address each concern point by point (in two parts due to character limits).
W1: Questionable practical value of OPSD. In my own experiments, OPSD has not yielded useful improvements, making me skeptical of its practical value in this setting.
We share the reviewer’s empirical observation: when applied naively, OPSD frequently stalls or even degrades accuracy, and diagnosing why naive OPSD fails and how to make it work reliably is the exact focus of our paper.
In our early exploration on
Qwen3-VL, OPSD also failed to give stable gains out of the box. For example, we initially observed that merely toggling the model’sthinkingmode on or off caused dramatic performance swings, consistent with recent findings in language OPSD showing that longthinking-enabled rollouts suffer from privilege-induced fork suppression and can turn gains into losses [1, 2]. That sensitivity led us to ask a more fundamental question: what can a privileged self-teacher actually teach the student, and when is privileged information genuinely learnable? Echoing recent studies on what privileged information works in OPSD [2, 3], we found that stronger or more explicit privilege does not automatically produce a better student; supervision fails whenever it is unreliable or ungrounded in what the student can observe (Table 2 & Table 7):T✗S✓) teacher failuresAlongside learnability routing, four practical choices in our
verlrecipe (Table 5) proved critical for stable gains (+4.90on 8B): (1) using concise evidence-first rollouts rather than unconstrained thinking traces, (2) optimizing student-centered Reverse KL, (3) retaining top-100 teacher logits with an aggregated tail bucket to avoid noisy tail matching, and (4) full-parameter tuning including the vision encoder for one epoch atlr = 1e-6. We will release our full code and recipes, and if the reviewer is willing to share their task, mode, or privilege setup, we would be glad to help analyze the differences.Q2: What motivates the use of OPSD?
Our motivation combines on-policy dense supervision, same-model alignment, and the early perceptual nature of video reasoning, isolated in Table 1:
SFT (Teacher)(+1.27), which suffers from exposure bias on teacher prefixes, OPSD supervises the student’s own rollouts on states visited at inference [4]. UnlikeGRPO(−0.01), which relies on sparse outcome rewards over 8 rollouts, OPSD gives much denser token-level guidance directly on the student rollout using only 1 rollout per prompt.Qwen3-VL-8B) and differ only in input views (sparse vs. sparse + dense GT frames), identical linguistic priors make capability transfer far more direct than from an external model [5].