acceptodds
Under review as a conference paper at ICLR 2027

Through the LENS of Learnability: Privileged Self-Distillation for Video Reasoning

Abstract

On-policy self-distillation (OPSD) has emerged as an effective post-training paradigm for video large language models (Video-LLMs), where a privileged self-teacher provides token-level guidance along student-generated trajectories. For video understanding, however, privileged information can take diverse forms, including answer hints, textual descriptions, and additional temporal observations, whose learning value is not captured by teacher accuracy alone. Through systematic analysis, we find that stronger teachers do not necessarily produce better students; whether the privileged evidence is actually learnable by the student is the key factor. We therefore revisit video OPSD through the lens of privilege learnability, governed by two factors: teacher reliability and student visibility. Based on this analysis, we propose LENS, a learnable privileged evidence framework for video OPSD. LENS constructs privileged teachers from densely grounded temporal evidence, estimates teacher reliability at the sample level, filters supervision beyond the student's observable scope, and routes each sample to distillation or reinforcement supervision based on teacher reliability. Extensive experiments across five challenging video reasoning benchmarks demonstrate that LENS consistently outperforms existing approaches, with the distilled student even surpassing its privileged teacher on temporal grounding tasks, validating the importance of learnability-aware privileged distillation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Video Understanding, On-Policy Self-Distillation, training framework

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_P19iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LENS, a video self-distillation framework that considers teacher reliability and student evidence visibility. A frozen teacher receives sparse global frames augmented with dense frames from annotated evidence intervals. Teacher correct samples receive reverse-KL distillation, while teacher wrong/student correct samples receive pairwise supervision. Training data are filtered by the student’s coverage of the evidence interval. Experiments on four videoQA benchmarks show improvements, particularly on the 8B backbone, with ablations examining routing, visibility filtering, and divergence objectives.

Critical strengths
  1. Useful analysis of privileged supervision. The paper distinguishes teacher accuracy from teaching utility and examines cases where additional visual evidence misleads the teacher.

  2. Practical handling of teacher failures. Pairing correct student responses with incorrect teacher responses avoids blindly imitating the teacher and contributes a 1.88 point gain in the 8B ablation, without adding inference-time requirements.

Critical weaknesses
  1. Questionable practical value of OPSD. In my own experiments, OPSD has not yielded useful improvements, making me skeptical of its practical value in this setting.

  2. Missing mainstream benchmark evaluation. The selected benchmarks omit widely used evaluations such as VideoMME, VideoMMMU, and MVBench. This makes it difficult to assess the baseline results and reported gains.

  3. Limited evidence for the learnability specific components. Visibility filtering contributes only 0.40 points, and reverse KL exceeds JSD by only 0.07 points. Gains over the strongest baseline are also small on 4B and 2B (0.57 and 0.34 points).

  4. Insufficient comparison with closely related methods like Video-OPSD.

Questions for authors
  1. Do the improvements persist on VideoMME, VideoMMMU, and MVBench under matched frame budgets and standard evaluation protocols?

  2. What motivates the use of OPSD?

4 comments
AuthorsOpenReview ↗

Response to Reviewer P19i (Part 1/2)

Initial Acknowledgment

We thank the reviewer for the constructive feedback and appreciate the recognition of our analysis distinguishing teacher accuracy from teaching utility, as well as the +1.88 gain from handling teacher failures without inference overhead. Below, we address each concern point by point (in two parts due to character limits).


W1: Questionable practical value of OPSD. In my own experiments, OPSD has not yielded useful improvements, making me skeptical of its practical value in this setting.

We share the reviewer’s empirical observation: when applied naively, OPSD frequently stalls or even degrades accuracy, and diagnosing why naive OPSD fails and how to make it work reliably is the exact focus of our paper.

In our early exploration on Qwen3-VL, OPSD also failed to give stable gains out of the box. For example, we initially observed that merely toggling the model’s thinking mode on or off caused dramatic performance swings, consistent with recent findings in language OPSD showing that long thinking-enabled rollouts suffer from privilege-induced fork suppression and can turn gains into losses [1, 2]. That sensitivity led us to ask a more fundamental question: what can a privileged self-teacher actually teach the student, and when is privileged information genuinely learnable? Echoing recent studies on what privileged information works in OPSD [2, 3], we found that stronger or more explicit privilege does not automatically produce a better student; supervision fails whenever it is unreliable or ungrounded in what the student can observe (Table 2 & Table 7):

Naive OPSD Setting in Our PaperObserved EffectSource
Blindly distilling privilege-confounded (T✗S✓) teacher failures−0.73 Avg. dropTable 2(c)
Using Forward KL instead of student-centered Reverse KL−0.93 below Base on CG-Bench; −2.45 vs. Reverse KL on QA-Ego4DTable 2(b)
Applying OPSD-Answer on 2B without CoT formatting capacity+0.02 (no meaningful gain)Table 7

Alongside learnability routing, four practical choices in our verl recipe (Table 5) proved critical for stable gains (+4.90 on 8B): (1) using concise evidence-first rollouts rather than unconstrained thinking traces, (2) optimizing student-centered Reverse KL, (3) retaining top-100 teacher logits with an aggregated tail bucket to avoid noisy tail matching, and (4) full-parameter tuning including the vision encoder for one epoch at lr = 1e-6. We will release our full code and recipes, and if the reviewer is willing to share their task, mode, or privilege setup, we would be glad to help analyze the differences.


Q2: What motivates the use of OPSD?

Our motivation combines on-policy dense supervision, same-model alignment, and the early perceptual nature of video reasoning, isolated in Table 1:

Method (8B)On-PolicySupervision SignalRolloutsΔ Avg.
SFT (GT)NoGold answer tokens----+0.37
SFT (Teacher)NoOff-policy teacher trace----+1.27
GRPOYesSparse outcome reward8−0.01
OPSD-AnswerYesDense token-level self-teacher1+2.71
LENS (Ours)YesDense token-level + reliability routing1+4.90
  1. On-policy dense supervision over SFT and RL. Unlike off-policy SFT (Teacher) (+1.27), which suffers from exposure bias on teacher prefixes, OPSD supervises the student’s own rollouts on states visited at inference [4]. Unlike GRPO (−0.01), which relies on sparse outcome rewards over 8 rollouts, OPSD gives much denser token-level guidance directly on the student rollout using only 1 rollout per prompt.
  2. Same-model alignment. Because teacher and student share the exact same checkpoint (Qwen3-VL-8B) and differ only in input views (sparse vs. sparse + dense GT frames), identical linguistic priors make capability transfer far more direct than from an external model [5].
  3. Early perceptual intervention in video reasoning. While on-policy RL aids video reasoning [6, 7], we observe that video errors typically start from misperceiving visual cues in opening tokens: 71.5% of repaired trajectories diverge within the first 20% of tokens (Figure 4, §4.4). Fixing early visual misreadings requires early intervention toward right visual grounding, where sparse end-of-sequence RL rewards are too indirect and dense guidance from an aligned privileged teacher is uniquely effective.

AuthorsOpenReview ↗

Response to Reviewer P19i (Part 2/2)

W2&Q1: Missing mainstream benchmark evaluation. The selected benchmarks omit widely used evaluations such as VideoMME, VideoMMMU, and MVBench. Do the improvements persist on VideoMME, VideoMMMU, and MVBench under matched frame budgets and standard evaluation protocols?

Our four primary benchmarks were chosen because they provide GT intervals [t_start, t_end] to measure coverage h(k)h(k) and starvation regimes (§4.4, Table 3). Within our compute budget, we directly evaluated our trained Qwen3-VL-8B LENS checkpoint from Table 1 (without retraining) against Base under matched frame budgets on Video-MME (w/o subs, 64f) and LSDBench [8] (32f), an hour-long video benchmark targeting short actions under sparse sampling:

Model (Qwen3-VL-8B)Video-MME (64f)LSDBench (32f)
Base61.755.3
LENS (Ours)62.9 (+1.2)57.6 (+2.3)

LENS improves over Base on both benchmarks (+1.2 on Video-MME, +2.3 on LSDBench) without retraining, confirming generalization to general and sparse-sampling long-video understanding. We will include these results in the revision.


W3: Limited evidence for the learnability specific components. Visibility filtering contributes only 0.40 points, and reverse KL exceeds JSD by only 0.07 points. Gains over the strongest baseline are also small on 4B and 2B (0.57 and 0.34 points).

Examined against their intended baselines, each result supports learnability and clear scaling behavior:

  1. Visibility goes beyond the final +0.40 step. On the teacher side (Table 8), keeping student-visible global frames (49.89) beats GT-only frames (Grounded, 48.81) by +1.08 (+2.20 on QA-Ego4D, +1.71 on S-EMBER). On the data side, gains follow a clear starvation gradient (+4.23 to +2.15, Figure 1d). Applied last in Table 2(a) after Reverse KL and routing (+1.88), visibility filtering still adds +1.30 on NExT-GQA (u16) and +0.61 on S-EMBER, diluted to +0.40 only by unweighted 5-column averaging yet remaining highly significant (95% CI [+0.28, +0.52], p=6.44×10−11p = 6.44 \times 10^{-11}; LENS vs. OPSD-Answer: +2.19, 95% CI [+1.67, +2.71], p=1.53×10−16p = 1.53 \times 10^{-16}).
  2. Reverse KL vs. Forward KL (and why JSD agrees). Standard OPSD uses mode-covering Forward KL (49.25), which penalizes unseen dense-frame modes and drops CG-Bench −0.93 below Base (31.96 vs. 32.89), whereas Reverse KL (49.89) gains +2.45 on QA-Ego4D and +1.19 on CG-Bench. JSD (β=0.5\beta=0.5, 49.82) is a bounded divergence incorporating student-weighted KL and removing Forward KL’s infinite mode-covering penalty; both beating Forward KL (+0.64 and +0.57) reinforces avoiding mode-covering over invisible evidence.
  3. Why gains narrow on 4B and 2B. LENS ranks best at all scales (+0.74 on 2B, +3.88 on 4B, +4.90 on 8B over Base), while smaller margins stem from: (i) fewer T✗S✓ samples on weaker backbones (as smaller students solve fewer sparse cases where the teacher errs), reducing self-reinforcement signal; and (ii) 2B instruction-following limits (Appendix C.1), requiring a letter-only prompt that bypasses early CoT tokens where 71.5% of repairs occur (Figure 4).

W4: Insufficient comparison with closely related methods like Video-OPSD.

Closely related video OPSD formulations, including concurrent Video-OPSD [9] and Clue-OPSD [10], are already evaluated in our paper (Table 1 & Table 8):

Method / Configuration (8B)Privileged InformationAvg.
Self-Distilled Reasoner [5] (OPSD-Answer, Table 1)GT answer49.98
Video-OPSD [9] / Clue-OPSD [10] (Grounded, Table 8)Dense GT frames only48.81
Contextualized Grounded Teacher (Ours, Table 8)Sparse + dense GT (T✓)49.89
LENS (Full Framework, Table 1)Sparse + dense GT + routing52.17

On matched T✓ distillation (Table 8), adding global sparse context (49.89) beats the GT-only Grounded teacher of Video-OPSD [9] and Clue-OPSD [10] (48.81) by +1.08. On all data, routing T✗S✓ failures lifts LENS to 52.17 (+3.36 over Grounded, +2.19 over OPSD-Answer [5]). We will explicitly discuss these comparisons in §5.


Final Acknowledgment

We hope our responses resolve your concerns and are happy to clarify further. Thank you again for your thoughtful review.


References

[1] Kaur et al., Rethinking OPSD for Thinking Models, 2026. [2] Zhang et al., What Does Privileged Information Add to OPSD?, 2026. [3] Tian et al., What Should a Self-Teacher See?, 2026. [4] Agarwal et al., On-Policy Distillation of LMs, ICLR 2024. [5] Zhao et al., Self-Distilled Reasoner, 2026. [6] Feng et al., Video-R1, NeurIPS 2025. [7] Sun et al., video-SALMONN-o1, ICML 2025. [8] Qu et al., LSDBench, ICCV 2025. [9] Wang et al., Video-OPSD, 2026. [10] Wang et al., Clue-OPSD, 2026.

Thanks for the response. However, mainstream benchmarks are still missing, and VideoMME alone is insufficient to demonstrate the effectiveness of the method. Therefore, I will maintain my score.

AuthorsOpenReview ↗

Additional Results on Mainstream Benchmarks

We thank the reviewer for the prompt feedback. Over the past few days, we have completed the remaining evaluations on MVBench and VideoMMMU. Below, we report the unified results across all four additional benchmarks.

Model (Qwen3-VL-8B)Video-MME (64f)MVBench (64f)VideoMMMU (64f)LSDBench (32f)
Base61.762.462.355.3
LENS (Ours)62.9 (+1.2)63.8 (+1.4)62.8 (+0.5)57.6 (+2.3)

LENS consistently improves over the Base model across all four benchmarks. Specifically, gains are most pronounced on LSDBench (+2.3), MVBench (+1.4), and Video-MME (+1.2), which rely on fine-grained temporal action perception and long-video evidence localization under sparse sampling, directly matching LENS’s design. Meanwhile, VideoMMMU focuses on lecture-style STEM disciplinary knowledge rather than temporal event grounding (with zero STEM data in our training set), so its gain is naturally smaller (+0.5) while still showing positive out-of-domain generalization without forgetting.

We will include these results in the revised paper, and hope they address your remaining concern.

Reviewer_nus4OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces LENS, an on-policy self-distillation method for video reasoning that addresses teacher errors and evidence unavailable to a sparse-frame student. A frozen teacher sees the student's uniformly sampled frames plus additional frames from a ground-truth evidence interval. LENS filters examples by student visibility, distills teacher-correct cases, and reinforces student-correct/teacher-wrong trajectories. Across four video QA benchmarks, LENS raises Qwen3-VL-8B average accuracy and show advantages.

Critical strengths
  1. The paper identifies a meaningful limitation of privileged distillation: additional evidence can improve a teacher while producing supervision that is unreliable or inaccessible to the sparse-input student.
  2. The two-path design is intuitive: it distills from teacher-correct cases and uses student-correct/teacher-wrong cases to reinforce the student’s own successful trajectory instead of copying a known error.
  3. The evaluation spans four video QA benchmarks and three model sizes. The reported 8B LENS results outperform the listed trained baselines on all four benchmark columns, and the ablations examine routing, visibility filtering, and the distillation divergence.
  4. The privileged frames and temporal annotations are used during training, while the evaluated student continues to use the benchmark’s ordinary sampled-frame input; evidence intervals are not required at inference time.
Critical weaknesses
  1. It seems the learnability signals rely on gold annotations and are imperfect proxies. The student's visibility is defined as the number of sampled frames whose timestamps fall inside a ground-truth interval, which does not establish that those frames contain the decisive visual evidence or that the student can interpret it. The teacher reliability is determined using the gold answer. Thus, the method seems to be a strongly supervised training recipe that depends on both temporal evidence annotations and answer labels. It would be better to have a discussion distinguish this setting from a generally applicable reliability or learnability estimator and discuss its annotation and scaling requirements.
  2. The authors (in Section 4.1) report that the 5.8K-example training set is dominated by cases with h(8) ∈ {1,2}, leaving unclear how well LENS generalizes to examples with no student-visible evidence or more complete coverage.
  3. The claimed student-over-teacher gains on NExT-GQA are only 0.29 and 0.56 points at 8 and 16 frames, respectively, so their statistical reliability is unclear.
Questions for authors
  1. How does LENS perform when h(8)=0 or h(8)>2?
  2. if there are no both-wrong examples in the training statistics, how those examples are handled in experiments?
  3. Are routing decisions recomputed from fresh rollouts during training?
2 comments
AuthorsOpenReview ↗

Response to Reviewer nus4

Initial Acknowledgment

We thank the reviewer for the positive assessment and for recognizing our two-path design, multi-scale gains, and annotation-free inference. Below, we address each point.


W1: Learnability signals rely on gold annotations and are imperfect proxies.

We agree: LENS is a privileged supervised recipe using training-only annotations ([t_start, t_end] and answers) as proxies, not an annotation-free runtime estimator (zero at inference). Following your suggestion of general estimators, we explored annotation-free options and will discuss three points in §3.2 and Limitations:

  1. Visibility estimator vs. GT intervals: On QA-Ego4D, GroundVQA [1] shows oracle GT intervals (53.5) beat self-predicted windows (48.7) and two-stage grounding (46.4), and Clue-OPSD [2] shows GT intervals beat random ones. Toward a visibility estimator, our base model’s zero-shot grounding was noisy, while strong grounding needs dedicated RLVR post-training (TimeLens [3], an external specialist). Though h(k)h(k) is a geometric proxy, Figure 1d and Table 3 confirm it tracks starvation.
  2. Reliability estimator vs. gold answers: Toward a label-free estimator, multi-rollout teacher agreement failed on T✗S✓, as dense adjacent frames biased the teacher toward the same distractor. Gold answers give an exact, zero-cost signal (we change "estimates" to "determines" in Abstract); learned estimators remain future work.
  3. Annotation cost & scaling: OPSD uses training-only privilege (CoTs [4], crops [5]). In video QA, [t_start, t_end] and answers come natively with grounded datasets (DynFrame, STAR, ActivityNet), needing far less effort than captions or an external teacher [6]. LENS gains +4.90 on 8B from 5,762 examples, using the same labels as SFT, GRPO, and OPSD-Answer.

W2&Q1: How does LENS perform when h(8)=0 or h(8)>2?

Trained on h(8)∈{1,2}h(8) \in \{1,2\} where headroom is largest, LENS generalizes strongly to both minimal-coverage and well-covered test questions (h>2h > 2).

  1. Training: Gains shrink as coverage grows (+4.23 to +2.15, Figure 1d). At h(8)=0h(8)=0, zero student frames fall in [t_start, t_end], risking hallucination; at h(8)>2h(8)>2, views overlap. In Table 2(a), filtering out h(8)=0h(8)=0 and over-covered cases lifts Avg. from 51.77 to 52.17 (p=6.4e-11p = 6.4\text{e-}11; +1.30 on NExT u16, +0.61 on S-EMBER).
  2. Evaluation (Table 3): LENS transfers to h > 2 buckets—gaining +5.32/+4.83 on NExT ((2,4]/(4,8]), +3.89/+3.57 on CG-Bench ((4,8]/>8), and +5.78/+4.46 on S-EMBER ((4,8]/>8)—and to (0,1] (+6.91 on NExT u8, +6.38 on Ego4D) via context frames (Appendix C.3).

W3: Statistical reliability of the 0.29 and 0.56 student-over-teacher margins on NExT-GQA.

We agree +0.29 and +0.56 are modest and will revise "surpasses" to "matches (and slightly edges out)." Crucially, the Teacher is evaluated as an oracle upper bound (k uniform + 32 dense GT frames), while the student sees only k uniform frames with zero GT timestamps (Table 1):

MetricNExT (u8)NExT (u16)Ego4DS-EMBERCG-Bench
S / T Frames8 / 4016 / 4864 / 9664 / 9664 / 96
Base / LENS Student66.95 / 73.0468.88 / 75.4045.70 / 51.5221.93 / 26.1832.89 / 34.70
Oracle Teacher (Upper Bound)72.7574.8455.5433.1448.69
Gap Closed105%109%59%38%11%

Matching this 40/48-frame oracle upper bound with 5× fewer frames shows T✗S✓ reinforcement goes beyond imitation.


Q2: Handling of both-wrong examples in experiments.

Both-wrong (T✗S✗) examples are discarded by design (Algorithm 1, line 13), as neither path applies when both fail. At the 6.3% T✗S✗ rate in Figure 1b (~390 cases vs. our 5,762 set), all were excluded before Table 4(a) (clarified in Appendix A).


Q3: Are routing decisions recomputed from fresh rollouts during training?

No: routing labels (T✓ vs. T✗S✓) are precomputed offline once, while distillation rollouts (T✓) stay online each step. Over one epoch (721 steps, lr = 1e-6), re-routing every 100 steps with the latest ckpt gives similar accuracy (52.01 vs. 52.17) at 1.4× cost:

Routing ModeAvg. (8B)Time
Offline + online rollout (Ours)52.171.0×
Online re-routing (per 100 steps)52.011.4×

We will clarify this in §3.2 and Algorithm 1.


Final Acknowledgment

We hope our responses resolve your questions and are glad to clarify further. Thank you for your review.


References

[1] Di & Xie, GroundVQA, CVPR 2024. [2] Wang et al., Clue-OPSD, 2026. [3] Zhang et al., TimeLens, CVPR 2026. [4] Zhao et al., SDR, 2026. [5] Yuan et al., Vision-OPD, 2026. [6] Li et al., Video-OPD, 2026.


Dear Reviewer,

Thank you again for your thoughtful comments and constructive suggestions. We are writing to kindly follow up on our responses to the points you raised.

As the discussion period progresses, we would greatly appreciate hearing whether our clarifications have addressed your concerns. If any questions remain, we would welcome the opportunity to provide further clarification or supporting evidence before the discussion closes.

Thank you for your time and consideration.

Best regards,
The Authors

Reviewer_NwuZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies on-policy self-distillation (OPSD) for video LLMs, where a frozen teacher conditioned on privileged evidence — the student's 8 sparse frames plus 32 dense frames sampled inside the ground-truth temporal interval — supervises a student that sees only the sparse frames. Through a preliminary study with four privilege forms, the authors argue teaching utility is governed not by teacher strength but by two factors: teacher reliability and student visibility.

Critical strengths

The privilege-learnability framing is a useful diagnostic, and the corrective/privilege-confounded partition is its sharpest instantiation. The observation that dense evidence can degrade a same-weights teacher on temporally adjacent events (7.2% of samples) is non-obvious and identifies a concrete mis-specification in standard OPSD: the teacher is not merely weaker there, it is anti-correlated with the correct answer. This matters because it reframes distillation from "imitate a better model" to "decide when imitation is licensed."

Critical weaknesses

Novelty is incremental and the comparison set is not the state of the art. Reverse KL versus forward KL is a standard mode-seeking/mode-covering trade-off, and reinforcing a correct self-trajectory against a wrong one is a pairwise preference objective. The closest prior work — Video-OPD, RP-OPSD, Vision-OPD, Video-OPDS, Self-Distilled Reasoner — is cited but never run as a baseline; comparisons stop at GRPO, SFT variants, and OPSD-Answer. "Consistently outperforms existing approaches" is therefore unsupported.

The headline claims are overstated relative to the numbers. Contextualized Grounded beats the Answer teacher by only +0.51 on average (Table 8) and loses on CG-Bench, so "a stronger teacher is not a better teacher" rests on a thin margin. The abstract's claim that the student surpasses its privileged teacher holds on only two of four benchmarks, by ≤0.65 points, while 4.8–7.0 point gaps remain on CG-Bench and S-EMBER — precisely the long/streaming regime the method is meant to serve.

Statistical and setting fragility. No seed variance is reported anywhere; several reported effects (+0.40 filtering, +0.64 reverse KL over forward KL) fall within plausible seed noise. Only one model family is tested, and the visibility filter consumes ground-truth temporal intervals at training time, which bounds deployment beyond benchmarks with dense annotations.

Questions for authors

see weakness

5 comments
AuthorsOpenReview ↗

Response to Reviewer NwuZ (Part 1/2)

Initial Acknowledgment

We thank the reviewer for the insightful feedback and appreciate the recognition that our analysis reframes distillation from “imitate a better model” to “decide when imitation is licensed,” identifying a non-obvious mis-specification in standard OPSD. Below, we address each concern point by point (in two parts).


W1(a): Novelty is incremental. Reverse KL versus forward KL is a standard mode-seeking/mode-covering trade-off, and reinforcing a correct self-trajectory against a wrong one is a pairwise preference objective.

We agree that reverse KL reflects a standard mode-seeking trade-off, which is not our claim, and would like to clarify that the pairwise margin loss designed to solve the unreliable-teacher issue through self-reinforcement is one of our contributions. However, rather than proposing a standalone loss formula, our paper makes two coupled contributions (§1):

  1. Our insight is that the privileged teacher really matters for student learning; we thus revisit privileged teachers in video OPSD through privilege learnability, governed by teacher reliability and student visibility.
  2. We propose LENS, pairing visibility filtering with dual-path reliability routing to convert teacher failures (T✗S✓) into a training signal via self-trajectory reinforcement (as shared weights make the student’s sparse trajectory a grounded positive).

Specifically, we study what form of privilege leaves learnable room for a sparse-frame student, when that supervision can be trusted, and how it shapes reasoning. Our central "non-obvious" finding is that under identical weights, denser GT evidence degrades the teacher on 7.2% of samples (T✗S✓) that the student already answers correctly from sparse frames, whereas standard OPSD blindly imitates the teacher here (Table 2c):

Supervision on T✗S✓ (n = 642)Avg. (8B)Δ vs. T✓ Only (49.89)
Distillation (imitate teacher)49.16−0.73
Self-trajectory reinforcement (LENS)51.77+1.88

W1(b): The closest prior work — Video-OPD, RP-OPSD, Vision-OPD, Video-OPDS, Self-Distilled Reasoner — is cited but never run as a baseline; comparisons stop at GRPO, SFT variants, and OPSD-Answer.

The OPSD methods applicable to video QA are already evaluated in our experiments: since concurrent Video-OPSD [2] and Clue-OPSD [3] have not released code or checkpoints, we reimplemented their privileged-teacher settings on our exact data and backbone (Appendix; Table 1 & Table 8):

MethodOPSDVideoPrivilege Setting & Avg. (8B)
Self-Distilled Reasoner [1]YesNoGT answer: 49.98 (OPSD-Answer, Table 1)
Video-OPSD [2] / Clue-OPSD [3]YesYesGT-interval frames: 48.81 (Grounded, Table 8)
Vision-OPD [4] / RP-OPSD [5]YesNo--
Video-OPD [6]NoYes--
LENS (Ours)YesYesSparse + GT frames: 49.89 (T✓) / 52.17 (Full)
  1. Video OPSD baselines: On T✓ distillation (Table 8), preserving global sparse context alongside dense GT frames (49.89) beats the reimplemented GT-only Grounded setting of Video-OPSD [2] and Clue-OPSD [3] (48.81) by +1.08. On all data, routing T✗S✓ errors lifts LENS (52.17) +2.19 over OPSD-Answer [1] and +3.36 over Grounded.
  2. Incomparable settings (--): Vision-OPD [4] and RP-OPSD [5] use image spatial privilege rather than video, while Video-OPD [6] distills a larger external teacher on timestamp grounding rather than self-distillation. We will rename these rows and scope our claims accordingly.

W2(a): Contextualized Grounded beats the Answer teacher by only +0.51 on average (Table 8) and loses on CG-Bench, so “a stronger teacher is not a better teacher” rests on a thin margin.

Table 8 is only an ablation on teacher-correct (T✓) data, whereas LENS also leverages privilege-confounded (T✗S✓) data—a subset naturally induced by our visual teacher:

Configuration (8B)Teacher Acc.T✓ DataT✗S✓ DataSourceAvg.
Semantic / Answer98.2% / 91.1%DistilledExcludedTable 849.05 / 49.38
Context. Grounded86.5%DistilledExcludedTable 849.89
OPSD-Answer91.1%DistilledDistilled (no routing)Table 149.98
Context. Grounded (Naive)86.5%DistilledDistilled (no routing)Table 2c49.16
LENS (w/o / w/ Filter)86.5%DistilledSelf-reinforcedTable 2c / 151.77 / 52.17

Even on T✓ data, 98.2% Semantic yields a weaker student than 86.5% Context. Grounded (49.05 vs. 49.89). Crucially, unlike answer teachers given the gold answer by construction, our visual teacher still reasons over frames and naturally induces fragile boundary disagreements (T✗S✓, 7.2%): distilling them hurts (49.16), whereas reinforcing them lifts LENS to 51.77 / 52.17, beating 91.1% OPSD-Answer (49.98) by +2.19.


AuthorsOpenReview ↗

Response to Reviewer NwuZ (Part 2/2)

W2(b): The abstract's claim that the student surpasses its privileged teacher holds on only two of four benchmarks, by ≤0.65 points, while 4.8–7.0 point gaps remain on CG-Bench and S-EMBER.

We clarify a key evaluation detail understated in Table 1’s caption: at test time, the Privileged Teacher serves as an oracle upper bound (k uniform + 32 dense GT-interval frames), whereas the LENS student sees only k uniform frames with zero GT timestamps:

MetricNExT (u8)NExT (u16)QA-Ego4DS-EMBERCG-Bench
Student / Teacher Frames8 / 4016 / 4864 / 9664 / 9664 / 96
Base / LENS Student66.95 / 73.0468.88 / 75.4045.70 / 51.5221.93 / 26.1832.89 / 34.70
Oracle Teacher (Upper Bound)72.7574.8455.5433.1448.69
Gap Closed105%109%59%38%11%

Under pure imitation, a blind 8-frame student is upper-bounded by a 40-frame oracle teacher; matching (and slightly edging out) this upper bound on NExT-GQA with 5× fewer frames is itself notable and stems from T✗S✓ self-reinforcement. On 27-minute CG-Bench videos, 57% of questions have h(64)≤1h(64) \le 1 (Table 3): when uniform frames miss short GT clues at test time, a remaining gap to an oracle-interval teacher is physically expected. We will revise the Abstract, §1, and §6 to state that the sparse student matches (and slightly edges out) its 40-frame privileged teacher on NExT-GQA while closing 38%–59% of the oracle gap on streaming and egocentric QA.


W3: Statistical and setting fragility. Only one model family is tested, and the visibility filter consumes ground-truth temporal intervals at training time, which bounds deployment beyond benchmarks with dense annotations.

1. Statistical significance & 5-benchmark averaging. Over all paired evaluations across our 5 benchmark settings (20,387 unique questions), we report 95% confidence intervals and two-sided Wald pp-values:

Comparison on Qwen3-VL-8BΔ Avg. (pp)95% CI (pp)Two-sided Wald p-value
LENS vs. Base+4.90[+4.29, +5.50]9.58×10−579.58 \times 10^{-57}
LENS vs. OPSD-Answer+2.19[+1.67, +2.71]1.53×10−161.53 \times 10^{-16}
+ Visibility Filter (Table 2a)+0.40[+0.28, +0.52]6.44×10−116.44 \times 10^{-11}

Even the +0.40 visibility filter step is highly statistically significant (p=6.44×10−11p = 6.44 \times 10^{-11}). Moreover, apparent small margins in Avg. stem from unweighted averaging across all 5 evaluation settings (including long-video CG-Bench where 57% of items have h(64)≤1h(64) \le 1), which dilutes substantial task-specific gains: Visibility Filter adds +1.30 on NExT-GQA (u16) and +0.61 on S-EMBER on top of routing (plus +1.08 in teacher design, Table 8), while Reverse KL beats Forward KL by +2.45 on QA-Ego4D and +1.19 on CG-Bench (where Forward KL drops −0.93 below Base).

2. Model family. Consistent with the experimental setting of closely related video distillation and RL works—including Video-OPSD [2], Clue-OPSD [3], Video-OPD [6], and Video-R1 [7], which likewise evaluate across model scales within the Qwen-VL family—we test three scales (2B, 4B, and 8B) of Qwen3-VL to strictly control visual tokenization and frame sampling across scales (we also note cross-family evaluation in Limitations).

3. Training-time temporal intervals. Both the teacher and filter use [t_start, t_end] strictly in training (zero annotations at inference): just as math OPSD uses CoTs [1] and image OPSD uses crops/high-res [4, 5], video QA naturally carries [t_start, t_end] intervals—needing no captions or external teacher [6] (+2.19 over OPSD-Answer).


Final Acknowledgment

We hope our responses resolve your concerns and are happy to clarify further. Thank you again for your thoughtful review.


References

[1] Zhao et al., "Self-Distilled Reasoner," arXiv:2601.18734, 2026.

[2] Wang et al., "Video-OPSD," arXiv:2608.27065, 2026.

[3] Wang et al., "Where to Look Matters," arXiv:2608.25356, 2026.

[4] Yuan et al., "Vision-OPD," arXiv:2605.18740, 2026.

[5] Zhu et al., "RP-OPSD," arXiv:2607.24447, 2026.

[6] Li et al., "Video-OPD," ICML, 2026.

[7] Feng et al., "Video-R1," NeurIPS, 2025.


Dear Reviewer,

Thank you again for your thoughtful review and constructive suggestions. We have posted our responses to your concerns.

When you have a chance, we would greatly appreciate your feedback on whether our responses address your concerns. If any points remain unclear or require further evidence, please let us know so that we can address them during the discussion period.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions and Discussion

Dear Reviewers and Area Chair,

We sincerely thank you for your thoughtful comments and constructive suggestions throughout the review and discussion process.

Following our detailed rebuttal responses, we have uploaded a revised manuscript (with updates highlighted in blue) incorporating the clarifications and additional experiments that address the concerns raised by Reviewers NwuZ, nus4, and P19i.

In particular, the revision includes:

  • Clarifications on related work and experimental protocols: To improve clarity, we explicitly connected our privilege configurations to closely related OPSD methods (Reviewers NwuZ, P19i), clarified the asymmetric oracle-teacher vs. sparse-student evaluation setup (Reviewers NwuZ, nus4), and refined the descriptions of data filtering, offline routing with online rollouts, and statistical significance tests (Reviewers nus4, NwuZ).
  • Additional mainstream and long-video benchmark evaluations: Directly addressing Reviewer P19i's main concern, we added zero-shot evaluations on Video-MME, MVBench, VideoMMMU, and LSDBench under matched frame budgets, demonstrating consistent gains across all eight evaluated video benchmarks without retraining.
  • Preservation of our core method and contributions: Aside from calibrating specific phrasing for precision, our core framework and empirical findings remain unchanged. Specifically, our work makes two coupled contributions: (1) showing that the form of privileged teacher critically governs student learning, and revisiting video OPSD through privilege learnability (teacher reliability and student visibility); and (2) proposing LENS, which integrates visibility filtering with dual-path reliability routing to convert privilege-confounded teacher failures (T✗S✓) into a discriminative training signal via self-trajectory reinforcement.

All corresponding updates and analyses have been incorporated into the main text and Appendices A–C.

As the discussion period is approaching its end, we would greatly appreciate any further feedback, particularly whether our responses and revisions have addressed your main concerns or if any questions remain.

Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.