acceptodds
Under review as a conference paper at ICLR 2027

OPD-TV: Rethinking Teacher Views for Privileged Supervision Transfer in Video On-Policy Distillation

Abstract

Privileged on-policy distillation extends on-policy distillation by allowing a teacher model to construct supervision from temporal evidence unavailable to the student. In Video OPD, this privilege is instantiated through additional temporal evidence available to the teacher while the student continues to operate under its original visual input. We find a supervision transfer gap: continuously enriching the teacher’s visual view improves the teacher’s grounding accuracy yet produces non-monotonic student gains. This supervision transfer gap arises because an enriched teacher can exploit visual cues unavailable to the student, introducing target shifts that the student cannot effectively absorb. To address this mismatch, we introduce OPD-TV, which regulates the transfer of privileged information based on token-level cross-view compatibility. By comparing the same teacher across both visual states, the proposed method explicitly isolates the additional corrections introduced by changing the teacher view from the standard policy correction in same-view distillation. It then anchors the transfer on the same-view teacher distribution and dynamically scales the transfer strength according to how closely the enriched teacher distribution remains aligned with the student-view teacher distribution. This continuous regulation selectively transfers useful visual refinements while suppressing corrections that are misaligned with the student’s visual input. Across three temporal grounding benchmarks, OPD-TV consistently improves privileged supervision transfer.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video on-policy distillation, multimodal learning, vision-language models, temporal video grounding.

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.40 over 5 reviews, above 68% of the others.
Reviewer_J2GvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies on-policy distillation for temporal video grounding. During training, the teacher sees frames from inside the annotated event, while the student (Qwen3-VL-8B) sees 128 uniform frames. The authors find that more event frames make the teacher better but, past a point, the student worse. OPD-TV runs the teacher under both views and scales the enriched view's correction by a per-token compatibility score c_t. It improves mIoU over Video OPD by 1.8, 5.5 and 6.2 points on the TimeLens versions of Charades, ActivityNet and QVHighlights.

Critical strengths
  1. An interesting observation about privileged teacher supervision. The paper identifies a meaningful supervision transfer gap: enriching the teacher's visual input can improve teacher accuracy without consistently improving student performance. The controlled experiments in Tables 2 and 4 help isolate the effect of privileged temporal evidence from teacher capacity. This is a useful finding, particularly because the required event annotations are already available in temporal grounding datasets.

  2. A simple method with promising empirical results. OPD-TV introduces a straightforward compatibility-based regulation mechanism with a closed-form distillation target. The ablation experiments in Table 3 provide evidence for the benefits of adaptive regulation over simpler alternatives. The reported improvements across three benchmarks, together with the analysis of failure cases and the released code, strengthen the practical value of the work.

Critical weaknesses
  1. Insufficient baseline comparisons and experimental reliability. Table 1 omits TimeLens-8B (Zhang et al., CVPR 2026), which uses the same Qwen3-VL-8B backbone with GRPO training and reportedly achieves an average mIoU of 58.0, nearly identical to OPD-TV's 58.1. Meanwhile, the paper's own GRPO baseline performs 3.6–5.4 points worse than TimeLens-8B, without sufficient details to explain the discrepancy. This raises concerns about the fairness of the comparisons. Furthermore, the additional improvement from compatibility regulation is relatively small (0.7–1.4 mIoU over direct privileged distillation and only 0.7 over constant weighting in Table 3). Since all results are based on single runs, their statistical reliability remains unclear. The apparent use of test-set performance for selecting configurations and checkpoints further complicates the interpretation of these gains. Together, these issues substantially weaken the empirical evidence supporting the method's superiority.

  2. Limited novelty and insufficient validation of the proposed mechanism. The supervision transfer gap is closely related to the previously studied imitation gap in asymmetric learning (Weihs et al., 2021; Warrington et al., 2021), which limits the conceptual novelty of the central observation. More importantly, the paper does not directly establish that the proposed compatibility score identifies supervision that the student cannot effectively learn. For example, it does not demonstrate whether lower compatibility corresponds to student frames missing the target event, while the failure cases in Figures 5 and 7 suggest that regulation cannot reliably address incorrect enriched-teacher supervision. The claimed monotonic improvement in teacher quality is also supported by a very small difference at higher enrichment levels (52.1 versus 52.2 in Figure 4). Finally, the evaluation is restricted to a single student architecture family and temporal grounding task, leaving the broader applicability of the proposed mechanism uncertain.

Questions for authors
  1. Baseline fairness: Could the authors include TimeLens-8B (Zhang et al., CVPR 2026) as a baseline and clarify the training configurations of all compared methods? In particular, why does the reported GRPO baseline substantially underperform TimeLens-8B despite using the same model backbone? A GRPO comparison trained on the same 2,400 examples would help establish whether the improvements can be attributed to the proposed distillation strategy.

  2. Statistical significance and model selection: Could the authors report mean and standard deviation over at least three independent runs for Direct Privileged OPD, uniform-weight transfer, and OPD-TV? Given the relatively small improvements attributable to compatibility regulation, this is important for assessing their reliability. Please also clarify whether the teacher view, frame budget, and checkpoints were selected using validation data or test-set results, and provide a consistent checkpoint selection protocol.

  3. Validation of the compatibility mechanism: What proportion of training examples have no overlap between the student's sampled frames and the annotated event interval? Does the proposed compatibility score ctc_t systematically decrease in these cases? A quantitative analysis relating event coverage, compatibility scores, and downstream grounding performance would help substantiate the claimed mechanism.

  4. Sensitivity to privileged-view construction: Do the K16 and K64 experiments reuse the training examples selected under the K32 configuration, or is the data selection performed independently for each setting? This distinction is important for interpreting the reported effects of teacher-view enrichment.

  5. Robustness to inference frame budgets: The student is trained using 128 uniformly sampled frames but evaluated at 2 FPS, potentially involving up to 768 frames. Do the reported improvements persist when the inference frame budget is fixed at 128 frames? This comparison would help determine whether the gains depend on the mismatch between training and inference conditions.

6 comments
AuthorsOpenReview ↗

Author Response (W1, Q1–Q2): Baselines and Statistical Reliability

We thank Reviewer J2Gv for the detailed review and for finding the supervision transfer gap a meaningful observation.

W1 & Q1: Baseline Fairness: Matched GRPO Training and Comparison with TimeLens-8B

The GRPO baseline in Table 1 uses the same Qwen3-VL-8B-Instruct base model and the same 2,400-example training sequence as OPD-TV. TimeLens-8B starts from the same base model and performs cold-start SFT on 30K examples. It then uses this model to select harder examples from the 100K training pool for thinking-free GRPO with batch-normalized policy optimization. We evaluate the released TimeLens-8B checkpoint under our Table 1 inference protocol at 2 FPS with a 768-frame cap. We will explicitly add TimeLens-8B to Table 1 to ensure a comprehensive comparison. TimeLens-8B reports 58.0 mIoU, and our evaluation yields 58.1 mean mIoU, matching OPD-TV (58.1). TimeLens-8B scores higher on Charades, whereas OPD-TV scores higher on ActivityNet and QVHighlights. TimeLens-8B uses 30K SFT examples and a 100K pool for data mining. OPD-TV reaches the same mean performance with only 2,400 examples and no SFT warm-up.

Tab. R1: Comparison with GRPO and TimeLens-8B under the Table 1 Inference Protocol (mIoU)

MethodCharadesActivityNetQVHighlightsMean mIoU
GRPO (Table 1)50.147.861.953.3
Same-view Video OPD50.249.161.453.6
Direct Privileged OPD51.353.466.257.0
OPD-TV52.054.667.658.1
TimeLens-8B55.353.465.658.1

Takeaway: OPD-TV matches TimeLens-8B under a common inference protocol.

Q2: Multi-Seed Statistics and the Configuration and Checkpoint Selection Protocol

  • Multi-seed results. We evaluate Direct Privileged OPD, uniform transfer, and OPD-TV across three random seeds (seeds 0, 1, and 2026) using the same ordered examples and update budget. OPD-TV consistently outperforms both baselines across all three seeds (Tab. R2).
  • Configuration selection protocol. The enriched-view event-frame allocation (K32K_{32}, Fig. 3) and the 128-frame budget (Table 6) were selected by evaluating Direct Privileged OPD on the evaluation benchmarks. The stopping point (step 125 of 150) was selected using Same-view Video OPD. Once determined, these hyperparameters were globally frozen and applied to OPD-TV and all baselines without any per-method tuning. As shown in Tab. R4, this optimal configuration is robust and persists even under independent and completely random data selection.
  • Training length and checkpoints. The matched OPD variants use the same 2,400-example sequence and are evaluated at the selected step 125, which OPD-TV reaches in 8.1 hours. Fig. 9 reports optimization efficiency over training time.

(Note: The table below reports three-seed means and standard deviations, whereas Table 3 in the main paper reports a single representative run under the default hyperparameters. Both settings show consistent gains over the two baselines.)

Tab. R2: Three-Seed Stability Comparison (Mean over Charades, ActNet, QVH)

MethodMean mIoU (±\pm std)[email protected] (±\pm std)
Direct Privileged OPD56.8 ±\pm 0.543.2 ±\pm 0.5
Uniform transfer (r=0.6r=0.6)57.3 ±\pm 0.443.7 ±\pm 0.4
OPD-TV58.1 ±\pm 0.444.7 ±\pm 0.4

OPD-TV consistently outperforms both baselines. The mean paired gains over Direct Privileged OPD are +1.3 ±\pm 0.2 mIoU and +1.5 ±\pm 0.2 [email protected]. The gains over uniform transfer are +0.8 ±\pm 0.1 mIoU and +1.0 ±\pm 0.2 [email protected]. Paired bootstrap comparisons yield p<10−4p < 10^{-4} against both baselines, pooling test examples across all three benchmarks. The adaptive-regulation gain therefore remains consistent across runs.

Takeaway: The regulation gain persists across training seeds under a shared configuration and checkpoint-selection protocol.

AuthorsOpenReview ↗

Author Response (W2, Q3): Compatibility and Transferability

W2 & Q3: Relation to the Imitation Gap and Validation of the Compatibility Mechanism

  • Relation to the imitation gap. Weihs et al. [1] and Warrington et al. [2] study the imitation gap arising from privileged observations in reinforcement-learning and POMDP settings. OPD-TV addresses this imitation gap in the token probability simplex of an autoregressive LLM. Unlike RL approaches, it requires no reward signal or auxiliary policy. Instead, it directly regulates token-level distribution shifts using cross-view support. We will discuss these conceptual connections in the related work.
  • Applicability to other tasks and architectures. The core formulation uses cross-view evaluation and structural target interpolation entirely within the token probability simplex. We evaluate it on temporal grounding with Qwen3-VL. The formulation makes no architecture-specific assumptions. It can therefore be applied to other settings with asymmetric visual observation, such as long-video QA and dense captioning.
  • Event coverage. Counting the uniformly sampled student frames that fall inside each annotated event shows that every training example has nonzero event coverage and 86.8% contain at least three in-event frames. Figure 2 (upper) illustrates the conceptual extreme of strict zero intersection to explain teleportation, whereas the observed training set has nonzero event coverage for every example.
  • Compatibility. Mean boundary-token compatibility reaches 0.94 on examples with at least three in-event frames and drops to 0.89 on examples with fewer than three, consistent with the 0.933 overall average in Fig. 4.
  • Downstream performance. Lower-compatibility groups contain fewer in-event student frames. Tab. R3 shows that OPD-TV improves over Direct Privileged OPD in the Mid (+1.3) and High (+0.2) groups. In the Low group, strong attenuation limits the transfer of some correct guidance. OPD-TV trails Direct by 0.8 mIoU but remains +1.4 mIoU above Same-view Video OPD.
  • Incorrect teacher corrections. Fig. 5 directly examines examples where the enriched-view teacher performs worse than the same-view teacher. Direct Privileged OPD incurs a 5.4-point mIoU drop, while OPD-TV reduces this drop to 2.3 points, mitigating more than half of the degradation. This shows that regulation reduces the damage from incorrect enriched-teacher supervision.
  • Teacher performance in Fig. 4. The teacher's accuracy plateaus at extreme enrichment levels. The difference between 52.1 and 52.2 is within evaluation variance (p=0.30p = 0.30). The K16K_{16} teacher is 2.0 mIoU below K32K_{32}. We will revise the manuscript to describe the teacher plateau and its implications for student transfer.

Tab. R3: Frame Coverage and mIoU across Compatibility Groups on QVHighlights (Fixed-128 Budget, Fig. 7 Groups)

GroupIn-Event FramesSame-viewDirect (vs. Same)OPD-TV (vs. Direct)
Low (cmin⁡<0.5c_{\min}<0.5)1.852.054.2 (+2.2)53.4 (−0.8)
Mid (0.5≤cmin⁡<0.90.5\le c_{\min}<0.9)3.558.061.5 (+3.5)62.8 (+1.3)
High (cmin⁡≥0.9c_{\min}\ge0.9)6.461.265.4 (+4.2)65.6 (+0.2)

Takeaway: Compatibility links student-visible event coverage to transfer quality and helps reduce damage from incorrect teacher corrections.

[1] L. Weihs et al., Bridging the Imitation Gap by Adaptive Insubordination, NeurIPS 2021.
[2] A. Warrington et al., Robust Asymmetric Learning in POMDPs, ICML 2021.

AuthorsOpenReview ↗

Author Response (Q4–Q5): Data Selection and Inference Budget

Q4: Whether the K16 and K64 Settings Reuse Examples Selected under K32

The K16K_{16} and K64K_{64} experiments in Fig. 3 reuse the 2,400 training examples mined under the default K32∪U96K_{32}\cup U_{96} teacher view. Keeping the training videos and queries fixed allows Fig. 3 to isolate the effect of the teacher view on the same training sequence.

To test whether the K32K_{32} peak depends on this shared selection, we evaluate two alternative training sets in Tab. R4. One setting mines 2,400 examples independently for each teacher view so that K16K_{16} and K64K_{64} no longer rely on examples selected by K32K_{32}. The other trains all three teacher views on the same duration-balanced Random-2400 sequence so that no teacher view influences data selection.

Tab. R4: Teacher-View and Data-Selection Controls (Mean mIoU over Charades, ActNet, QVH)

Teacher ViewShared K32K_{32}IndependentRandom-2400
K16∪U112K_{16}\cup U_{112}54.755.154.5
K32∪U96K_{32}\cup U_{96}57.057.056.4
K64∪U64K_{64}\cup U_{64}54.354.654.1

Takeaway: The preference for intermediate enrichment persists when examples are selected independently for each teacher view.

Q5: Robustness under a Fixed 128-Frame Inference Budget

We evaluate the same checkpoints using 128 uniformly sampled frames at inference to match the student's training view. Tab. R5 reports the results across all three benchmarks.

(Note: Tab. R5 uses 128 uniformly sampled frames for all methods, while Table 1 uses 2 FPS sampling with up to 768 frames. OPD-TV retains the highest mean mIoU under this fixed inference budget.)

Tab. R5: Fixed-128 Inference with Matched Training and Inference Frame Budgets (mIoU)

MethodCharadesActivityNetQVHighlightsMean mIoU
Same-view Video OPD50.147.258.952.1
Direct Privileged OPD53.648.362.654.8
OPD-TV53.651.263.156.0

Takeaway: The grounding gains persist when training and inference use the same frame budget.

Reviewer_SEwWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the problem of Privileged On-Policy Distillation in Video OPD, where the teacher is provided with richer input information than the student. The authors observe a phenomenon called the Supervision Transfer Gap, where providing the teacher with richer views improves the teacher's grounding accuracy but does not improve performance when distilled into the student. The authors propose OPD-TV, which adaptively adjusts the influence of richer teacher views based on Token-level Cross-view Compatibility, leveraging the advantages of additional visual information while preventing indiscriminate supervision shifts caused by the additional views. On Temporal Grounding Benchmarks, OPD-TV achieves higher performance than the baselines, Same-view Distillation and Direct Privileged Transfer.

Critical strengths

The figure illustrating the setting in the Preliminaries section was appropriate for helping readers understand the paper. The paper is clearly structured and easy to read. In particular, the analysis presented in Section 4.3 was clear and well-suited to supporting the paper's claims.

Critical weaknesses

[Minor 1] In the caption of Figure 2, the labels (left) and (right) are incorrect and should be changed to (upper) and (lower) to match the actual layout of the figure.

[Major 1] Is Cross-view Compatibility, defined in Section 3.2, the most appropriate signal? I understand that the proposed method distills the View Correction induced by Additional Frames more strongly when Compatibility is high, and more weakly when Compatibility is low. However, this signal design raises two concerns.

(1) When a token that originally had a low probability under the Same-view Teacher receives a high probability due to additional frames, that token may be important for producing the correct answer. If this token can also be sufficiently inferred from the Student's original visual input, the corresponding distillation signal could be useful. Therefore, the proposed Cross-view Compatibility does not appear to distinguish between such cases and target shifts that are difficult for the Student to learn.

(2) In addition to Cross-view Compatibility, there could be various ways to consider the distributional differences between the Same-view Teacher and the Enriched-view Teacher. For example, measuring a simple distributional difference, such as KL divergence, and optimizing a threshold to adjust the Transfer Strength could also serve as a meaningful signal for supervision transfer.

I would appreciate further discussion regarding (1) and comparative experiments regarding (2).

[Major 2] The proposed method adjusts the degree of transfer of View Correction induced by additional frames based on tokens that have high probabilities under the Student's original uniformly sampled frames. Therefore, this method appears to work effectively when the frames provided to the Student contain sufficient information to solve the target task to some extent.

This naturally raises a question: In long-video tasks, uniform sampling becomes increasingly sparse under a limited frame budget, and the sampled frames may not contain information relevant to the target task. Would the proposed method be difficult to extend to such long-horizon tasks?

Although the authors provide experiments with different frame budgets, the analysis of cases where the video length increases or the Student's sampled frames do not contain the ground-truth event is insufficient. Therefore, it would be valuable to provide further discussion on this issue or additional evaluations on long-video benchmarks.

[Major 3] Section 4.4 evaluates the effectiveness of Adaptive Regulation across generation steps, but the performance improvement is limited. Compared to Uniform Transfer Strength, which is the most important baseline in this ablation study, the improvement in mIoU is only 0.7 percentage points. Experiments with multiple random seeds are needed to verify whether the performance difference is statistically meaningful.

Questions for authors

The questions are summarized below. Please refer to the Critical Weaknesses section for detailed explanations.

Major 1: Is Cross-view Compatibility the most appropriate signal for regulating supervision transfer, and how does it compare with simpler distribution-based alternatives, such as KL divergence?

Major 2: Can the proposed method remain effective on long-video tasks when the Student's sampled frames do not contain the ground-truth event?

Major 3: Is the 0.7 percentage points mIoU improvement over Uniform Transfer Strength statistically significant across multiple random seeds?

5 comments
AuthorsOpenReview ↗

Author Response (Major 1): Compatibility and Alternative Controls

We thank Reviewer SEwW for the positive assessment and the thoughtful questions.

Major 1: Compatibility versus Distribution-Distance Controls

Cross-view compatibility determines how strongly to transfer a privileged correction:

  • High compatibility: the extra frames mainly redistribute probability among boundaries already plausible under the student view, so the correction is transferred more strongly.
  • Low compatibility: the extra frames move probability toward boundaries with little support under the student view, so the correction is transferred more cautiously.

A low-compatibility correction can still be accurate and learnable. On boundary tokens with ct<0.5c_t < 0.5, the enriched-view teacher moves closer to the ground truth than the same-view teacher in 83% of cases. The Low group in Fig. 6 has a mean outside mass of 78.8%, giving cˉt≈0.21\bar{c}_t \approx 0.21. Under rt=0.2+0.8ctr_t=0.2+0.8c_t, this yields a mean transfer weight of rˉt≈0.37\bar{r}_t \approx 0.37, so these corrections retain nonzero weight. The Truncate-low ablation in Table 3 supports this choice: discarding low-compatibility corrections reduces the three-benchmark mean mIoU from 58.1 to 55.5.

Compatibility versus KL/JSD. KL divergence is asymmetric, but its scalar magnitude does not identify whether a change stays within the same-view support. Both sharpening among supported candidates and shifting probability toward unsupported candidates can produce a large KL divergence. Compatibility instead measures the enriched-view teacher's probability mass retained within the same-view support, distinguishing support retention from a shift beyond that support.

Keeping the training sequence and update budget fixed, we compare OPD-TV against two distribution-distance controls (Tab. R1). Both gates use the same minimum weight of 0.2.

  • KL-weighted transfer: Uses the forward KL divergence from the enriched teacher to the same-view teacher:

    gt=σ(τKL−DKL(q~t∥qt)).g_t = \sigma(\tau_{\mathrm{KL}} - D_{\mathrm{KL}}(\tilde q_t \parallel q_t)).

  • JSD-distance gate: Uses the symmetric JSD between the two teacher distributions:

    gt=max⁡(0,1−DJSD(qt∥q~t)τJSD).g_t = \max\left(0, 1 - \frac{D_{\mathrm{JSD}}(q_t \parallel \tilde q_t)}{\tau_{\mathrm{JSD}}}\right).

For both gates, the transfer weight is rt=0.2+0.8gtr_t = 0.2 + 0.8g_t. We report the best result for each control over its tested parameter settings.

Tab. R1: Distribution-Distance Controls: KL and JSD (Mean over Charades, ActNet, QVH)

MethodMean mIoU[email protected]
KL-weighted transfer57.343.7
JSD-distance gate57.544.0
Uniform transfer (r=0.6r=0.6)57.443.8
OPD-TV58.144.7

Both distribution-distance controls remain below OPD-TV's 58.1 mean mIoU, supporting the benefit of measuring cross-view support directly.

Takeaway: Soft compatibility regulation retains potentially useful corrections and outperforms the tested distribution-distance gates.

AuthorsOpenReview ↗

Author Response (Major 2–3, Minor 1): Sparse Evidence, Stability, and Figure Correction

Major 2: Long Videos and Sparse Event Coverage

Current temporal grounding benchmarks contain mostly moderate-length videos, so uniformly sampling 128 student frames usually preserves some evidence from the target event. Every training example contains at least one uniformly sampled student frame inside the annotated event, and 86.8% contain at least three. Mean boundary-token compatibility is 0.94 when at least three event frames are visible and 0.89 when fewer than three are visible. The training examples therefore present a challenge of sparse event evidence.

We evaluate the same checkpoints on ActivityNet under the Fixed-128 inference protocol and group test examples by video duration. We also examine queries whose 128 uniformly sampled frames miss the annotated event (n=80n = 80 of the 4,500 ActivityNet test queries). Zero overlap under 128 uniform frames occurs only when an event is extremely short (less than 1/1281/128 of the video duration). Consequently, absolute mIoU is naturally low across all models (base model: 5.1 mIoU). OPD-TV achieves a +1.0 mIoU gain over Direct Privileged OPD on this zero-overlap subset (8.6 vs. 7.6). Moreover, on the longest third of ActivityNet videos (≥176\ge 176s), OPD-TV reaches 47.6 mIoU, substantially improving upon the base model (42.5) and exceeding TimeLens-8B (45.4).

Event coverage versus contextual cues. Visual masking removes image content, while temporal sampling can miss the event and still retain surrounding context. In the practical zero-overlap subset (n=80n=80), the 128 sampled frames happen to miss the exact GT interval, but the student can still observe surrounding contextual frames. OPD-TV regulates the teacher's target shifts while the student learns from these contextual frames. It retains a +1.0 mIoU gain over Direct Privileged OPD even without explicit event evidence.

Tab. R2: Results by Video Duration and Event Coverage (ActivityNet, Fixed-128 Protocol)

SubsetSame-viewDirectOPD-TVGain over Direct
Shortest third51.553.054.5+1.5
Middle third47.047.851.5+3.7
Longest third (≥176\ge 176s)43.144.147.6+3.5
Zero frames inside the event (n=80n=80)6.87.68.6+1.0

Takeaway: OPD-TV retains its advantage on longer videos and on examples where sparse sampling misses the event.

Major 3: Statistical Significance of the Gain over Uniform Transfer across Seeds

We evaluate Direct Privileged OPD, uniform transfer, and OPD-TV across three random seeds (seeds 0, 1, and 2026) using the same ordered examples and update budget (Tab. R3). The uniform coefficient is fixed at r=0.6r=0.6, the best constant setting in a coarse grid over r∈{0.4,0.6,0.8}r\in\lbrace 0.4,0.6,0.8\rbrace.

(Note: The table below reports three-seed means and standard deviations, whereas Table 3 in the main paper reports a single representative run under the default hyperparameters. Both settings show consistent gains over the two baselines.)

Tab. R3: Three-Seed Stability Comparison (Mean over Charades, ActNet, QVH)

MethodMean mIoU (±\pm std)[email protected] (±\pm std)
Direct Privileged OPD56.8 ±\pm 0.543.2 ±\pm 0.5
Uniform transfer (r=0.6r=0.6)57.3 ±\pm 0.443.7 ±\pm 0.4
OPD-TV58.1 ±\pm 0.444.7 ±\pm 0.4

OPD-TV consistently outperforms both baselines. The mean paired gains over Direct Privileged OPD are +1.3 ±\pm 0.2 mIoU and +1.5 ±\pm 0.2 [email protected]. The gains over uniform transfer are +0.8 ±\pm 0.1 mIoU and +1.0 ±\pm 0.2 [email protected]. Paired bootstrap comparisons yield p<10−4p < 10^{-4} against both baselines, pooling test examples across all three benchmarks. The adaptive-regulation gain therefore remains consistent across runs.

Takeaway: The gain over uniform transfer is consistent across training seeds and supported by the paired comparisons.

Minor 1: Panel Labels in the Figure 2 Caption

Thank you for catching this. We will correct the labels (left) and (right) to (upper) and (lower) in the Fig. 2 caption to match the layout.

With the discussion period closing tomorrow, we would greatly appreciate it if you could revisit our rebuttal and the revised manuscript. Please let us know if our response addresses your concerns, or if there are any remaining points we can clarify. Thank you very much for your time!

Reviewer_sN7fOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies privileged on-policy distillation for video temporal grounding, where the teacher receives additional visual evidence unavailable to the student. It shows that a stronger teacher view does not always lead to better student learning because privileged evidence can introduce supervision that is incompatible with the student’s visual input. To address this, the paper proposes OPD-TV, which regulates privileged teacher corrections using cross-view compatibility and improves performance across multiple temporal grounding benchmarks.

Critical strengths
  • Insightful analysis of privileged teacher supervision: The paper shows that giving the teacher richer visual information does not always lead to better student learning. Its analysis of this supervision transfer gap provides a clear motivation for the proposed method.

  • Strong experimental validation: The paper provides controlled ablations that separate the effect of privileged teacher views from the proposed compatibility regulation, making the source of the performance gains relatively clear.

  • Consistent empirical improvements: OPD-TV shows consistent gains across multiple temporal grounding benchmarks and teacher–student configurations, supporting the robustness of the proposed approach.

Critical weaknesses
  • Limited motivation for the privileged teacher view: The paper shows that giving the teacher query-relevant event frames can improve its predictions, but it is unclear why a visual input gap between the teacher and student is necessary. Such a gap may also encourage the student to rely less on its observed frames and more on non-visual priors, potentially leading to visual neglect issues [A]. The paper would benefit from a clearer motivation for introducing this gap and further analysis of what specific benefits it provides to student learning.

[A] Wu et al., Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images, ACL 2025.

  • Unclear whether the student remains grounded in its observed visual evidence: OPD-TV transfers supervision from privileged frames that the student never observes. When these frames are important for the teacher’s prediction, it is unclear how the student learns to make the same prediction from its limited visual input. The student may learn useful contextual cues, but it may also rely more on language or dataset-specific priors (i.e., visual forgetting issues [B]), leading to accurate but weakly grounded predictions on visual evidence and potentially increasing hallucination. Additional analysis, such as frame-level visual attribution or performance when relevant frames are masked, would help verify whether the gains remain visually grounded.

[B] Sun et al., Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning, ACL 2025.

  • Limited analysis of what the student learns from privileged supervision: The paper shows that privileged teacher evidence improves downstream grounding performance, but provides limited analysis of why the student improves despite not observing the privileged frames. It remains unclear whether the student learns better temporal reasoning from its available visual context, improves its use of relevant frames, or simply exploits dataset-specific priors. Further analysis of the student’s reasoning behavior and visual reliance would help clarify the mechanism behind the gains.

  • Potential selection bias from evidence-sensitive training examples: The training data are intentionally selected based on how much the teacher’s prediction changes under the enriched visual view. This selection criterion is closely aligned with the setting where the proposed privileged supervision is expected to be most useful, which may bias the training distribution toward method-favorable examples. Moreover, the definition of an evidence-sensitive example depends on the behavior of a specific mining model, raising questions about whether the same examples would remain informative for different teacher or student models. Although the paper includes a random-sampling control, further evaluation on naturally sampled data and under cross-dataset distribution shifts would better establish the generality of the proposed method.

  • Figure 2 is difficult to interpret: The distinction between “teleportation” and “sharpening” is not immediately clear. Simplifying the figure and more clearly highlighting the roles of the student view, privileged frames, and OPD-TV would improve readability.

  • Unclear whether cross-view compatibility truly measures transferability: OPD-TV assumes that corrections that remain close to the same-view teacher distribution are more transferable to the student. However, distributional agreement does not necessarily imply that the correction is useful or learnable from the student’s visual input. A stronger analysis comparing the proposed compatibility score with actual downstream transfer quality would better justify this key assumption.

  • Higher computational cost than naive OPD is not sufficiently analyzed: OPD-TV requires paired teacher evaluations under both the same-view and enriched-view inputs, which likely introduces additional training-time computation and memory overhead compared with standard Video OPD. Although the paper reports wall-clock efficiency relative to GRPO, a direct comparison of training cost, GPU hours, memory usage, and performance gain against naive OPD or Direct Privileged OPD would be helpful to better assess the practical trade-off of the proposed method.

Questions for authors
  • Applicability to self-distillation settings: Can the proposed cross-view compatibility mechanism also be applied to self-distillation settings, where the teacher and student share the same backbone but receive different amounts or types of visual information? Since OPD-TV is motivated by an information asymmetry between teacher and student views, it would be interesting to understand whether the same regulation strategy remains effective when the teacher advantage comes primarily from richer visual inputs rather than larger model capacity.
6 comments
AuthorsOpenReview ↗

Author Response (W1–W3): Privileged Views and Visual Grounding

We thank Reviewer sN7f for recognizing our analysis of the supervision transfer gap and respond to each concern below.

W1–W3: Motivation for the Privileged Teacher View and Visual Grounding of the Student

1. Why introduce an enriched teacher view? Recent multimodal OPSD methods, including Vision-OPD [1], ViCuR [2], and Visual-Advantage OPD [3], have already demonstrated the effectiveness of privileged visual supervision under asymmetric views. In our temporal grounding setting, the privileged view exposes the teacher to denser query-relevant event evidence while keeping the visual budget fixed at 128 frames, producing stronger grounding supervision without increasing input capacity.

We verify that this teacher advantage comes from visual evidence rather than the denser timestamp pattern induced by event-enriched sampling. Shuffling or masking the event images reduces teacher performance, while replacing all frames with black images causes both views to collapse. The enriched sampling pattern retains only a small residual advantage when visual content is removed.

Tab. R1: Teacher-Side Visual and Timestamp Controls

(a) Event-image perturbation (Enriched-view teacher: K32∪U96K_{32}\cup U_{96})

Evaluation SettingSame-view Teacher (U128U_{128})Clean EnrichedShuffled EnrichedMasked Enriched
Teacher mIoU49.452.147.843.5

(b) Timestamp-only control with all-black frames

Teacher ViewU128U_{128}K32∪U96K_{32}\cup U_{96}Difference (95% CI)
Teacher mIoU7.112.3+5.2 [4.2, 6.3]

These controls show that the enriched-view teacher gains primarily from task-relevant event images rather than timestamp density alone.

2. Does the student remain grounded in its observed visual evidence? The view gap is relatively mild: 86.8% of the training examples contain at least three in-event student frames, and every example has nonzero event coverage. The enriched teacher view therefore densifies task-relevant temporal evidence around frames the student already observes.

We evaluate the same QVHighlights queries with the original student frames, masked event frames, and an equal number of masked background frames while fixing the query, timestamps, and decoding.

Tab. R2: Visual-Evidence Masking on QVHighlights (Default 2 FPS Protocol)

MethodOriginalEvent MaskedBackground Masked
Qwen3-VL-8B-Instruct59.434.256.4
Same-view Video OPD61.434.858.5
Direct Privileged OPD66.235.663.1
OPD-TV67.635.164.7

Masking event frames causes a substantially larger performance drop than masking the same number of background frames across all methods, showing that OPD-TV remains dependent on task-relevant evidence in the student's own view.

3. What does the student learn from privileged supervision? Direct Privileged OPD transfers every enriched-view correction equally, including shifts toward boundaries weakly supported by the student view. OPD-TV instead transfers refinements within supported candidates more strongly and attenuates shifts beyond them. In Tab. R2, OPD-TV's advantage over Direct Privileged OPD persists with the original frames and under background masking, but disappears when the event frames are removed. This intervention shows that OPD-TV improves the student's use of available event evidence.

Visual neglect [4] and visual forgetting [5]. Privileged targets risk encouraging the student to ignore its visual input (visual neglect) or memorize dataset biases (visual forgetting). OPD-TV actively mitigates these risks by anchoring the transfer strength to the student's observed view. As shown in Tab. R2, masking the event frames causes a severe performance drop (from 67.6 to 35.1 mIoU). These results show that the OPD-TV student actively uses task-relevant visual evidence.

Takeaway: Privileged supervision improves grounding while the student continues to rely on its observed visual evidence.

[1] Q. Yuan et al., Vision-OPD: Learning to See Fine Details for Multimodal LLMs via On-Policy Self-Distillation, 2026.
[2] K. Tian et al., ViCuR: Visual Cues as Recoverable Privilege for Multimodal On-Policy Distillation, 2026.
[3] R. Liu et al., Visual-Advantage On-Policy Distillation for Vision-Language Models, 2026.
[4] S. Wu et al., Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images, ACL 2025.
[5] H.-L. Sun et al., Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning, ACL 2025.

AuthorsOpenReview ↗

Author Response (W4–W5): Data Selection and View Corrections

W4: Selection Bias from Evidence-Sensitive Training Examples and Cross-Dataset Generality

Following Video-OPD [6], we use evidence-sensitive sampling to focus training on examples where visual information changes the model's grounding prediction. Appendix D.4 evaluates OPD-TV on both the evidence-sensitive sequence and a duration-balanced Random-2400 sequence across Charades, ActivityNet, and QVHighlights. Random sampling lowers the three-benchmark mean mIoU from 58.1 to 57.5, as reported in Appendix D.4. OPD-TV trained on Random-2400 still exceeds Direct Privileged OPD trained on the evidence-sensitive sequence on all three benchmarks (51.4 / 54.3 / 66.8 vs. 51.3 / 53.4 / 66.2 mIoU). OPD-TV remains effective under random selection, with evidence-sensitive selection providing an additional gain.

Training source and cross-benchmark evaluation. Our training examples come from TimeLens-100K (Appendix D). Its sources are disjoint from Charades, ActivityNet, and QVHighlights, where we evaluate the resulting model. All reported results are therefore cross-dataset, which complements the Random-2400 control by assessing transfer across multiple benchmark distributions.

We further train Direct Privileged OPD on exactly the same Random-2400 sequence with identical initialization and optimization settings.

Tab. R3: Compatibility Regulation under Evidence-Sensitive and Random Sampling (Mean mIoU over Charades, ActNet, QVH)

Training-Pair SelectionDirectOPD-TVGain over Direct
Evidence-sensitive57.058.1+1.1
Random-240056.457.5+1.1

OPD-TV preserves the same gain over direct transfer under both selection protocols, demonstrating that compatibility regulation remains effective regardless of the selection protocol. Table 5 further evaluates enriched-view supervision under 4B-to-8B OPD, 8B-to-8B OPSD, and 32B-to-8B OPD. The student improves across all three teacher configurations and grounding benchmarks, establishing robustness across teacher capacities and provenance.

Dependence on the mining model. The Random-2400 control (Tab. R3) uses no mining model at all, and OPD-TV retains its +1.1 mIoU advantage there. The regulation gain therefore does not depend on the examples selected by a specific mining model.

Takeaway: The gains across the evaluated benchmarks persist under random sampling without dependence on mining-model selection.

W5: Interpreting Figure 2: Sharpening versus Teleportation

Fig. 2 distinguishes whether the extra teacher frames refine a boundary already plausible under the student view or shift probability toward a boundary supported mainly by the privileged view. We refer to these two cases as sharpening and teleportation. OPD-TV assigns greater weight to sharpening because the student observes relevant visual evidence, while attenuating teleportation where student support is limited. Figure 2 (upper) illustrates the conceptual extreme of strict zero intersection to explain teleportation, whereas the observed training set has nonzero event coverage for every example. We will update Fig. 2 to illustrate "Sparse/No intersection" and correct the "OPD-TA" typo to "OPD-TV".

Takeaway: Sharpening refines supported predictions, while teleportation shifts probability toward predictions with little same-view support.

[6] J. Li et al., Video-OPD: Efficient Post-Training of Multimodal Large Language Models for Temporal Video Grounding via On-Policy Distillation, 2026.

AuthorsOpenReview ↗

Author Response (W6–W7): Transferability and Training Cost

W6: Whether Cross-View Compatibility Tracks Downstream Transferability

Cross-view compatibility measures the degree of support for an enriched-view correction under the student view. It provides a graded signal for regulating transfer. We validate this signal against both the visual evidence available to the student and the realized downstream benefit of privileged transfer.

Tab. R4: Frame Coverage and mIoU across Compatibility Groups on QVHighlights (Fixed-128 Budget, Fig. 7 Groups)

GroupIn-Event FramesSame-viewDirect (vs. Same)OPD-TV (vs. Direct)
Low (cmin⁡<0.5c_{\min}<0.5)1.852.054.2 (+2.2)53.4 (−0.8)
Mid (0.5≤cmin⁡<0.90.5\le c_{\min}<0.9)3.558.061.5 (+3.5)62.8 (+1.3)
High (cmin⁡≥0.9c_{\min}\ge0.9)6.461.265.4 (+4.2)65.6 (+0.2)

Compatibility increases with the amount of event evidence visible to the student, and the downstream benefit of privileged supervision increases from the Low to the High group. Direct privileged transfer improves over Same-view Video OPD by 2.2, 3.5, and 4.2 mIoU across the three groups, while OPD-TV achieves positive gains over Direct Privileged OPD in both the Mid (+1.3) and High (+0.2) groups. In the Low group, strong attenuation limits the transfer of some correct guidance. OPD-TV trails Direct by 0.8 mIoU but remains +1.4 mIoU above Same-view Video OPD.

Low-compatibility corrections can still be useful. Both privileged methods outperform Same-view Video OPD in the Low group (+2.2 and +1.4 mIoU). Removing these corrections in the Truncate-low ablation reduces the overall mIoU from 58.1 to 55.5. OPD-TV therefore uses compatibility to continuously regulate transfer strength while retaining useful guidance across the compatibility spectrum.

We will revise the abstract to state that OPD-TV "attenuates corrections with low cross-view support" instead of "suppressing corrections that are misaligned with the student's visual input." We will also explicitly discuss the mixed outcomes of the low-compatibility group in the experimental analysis section of the main text.

Takeaway: Compatibility is associated with downstream transfer benefit, supporting its use as a soft regulation signal.

W7: Training Cost Relative to Same-View and Direct Privileged OPD

We compare training costs over the same 125 updates on 8 ×\times A800-80GB GPUs. Same-view Video OPD and Direct Privileged OPD take 6.8 and 7.4 hours, respectively. OPD-TV takes 8.1 hours. The additional teacher-view evaluation adds 0.7 hours (~9%) over Direct Privileged OPD. This remains substantially faster than the 14.2 hours required by GRPO in Appendix C.2. Peak memory changes little because the teacher passes run sequentially.

(Note: The Same-view Video OPD baseline in Tab. R5 uses an 8B teacher and corresponds to the 53.6 mean mIoU result in Table 1. The same-view baseline in Table 4 uses a 32B teacher under identical optimization budgets and achieves 53.9 mean mIoU, labeled as 54.0 in Fig. 3.)

Tab. R5: Training Cost Comparison (8 ×\times A800 GPUs, 125 Steps; Mean mIoU over Charades, ActNet, QVH)

MethodTimeGPU HoursPeak MemoryMean mIoU
Same-view Video OPD6.8 h54.468.2 GB53.6
Direct Privileged OPD7.4 h59.268.2 GB57.0
OPD-TV8.1 h64.868.5 GB58.1

Takeaway: OPD-TV improves on direct privileged transfer with modest additional training time and little change in peak memory.

Reviewer_u737OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies privileged on-policy distillation for temporal video grounding, where the teacher receives event-enriched video frames while the student generates trajectories from uniformly sampled frames. The authors identify a mismatch between teacher accuracy and teaching effectiveness: stronger event enrichment can improve the teacher while degrading the distilled student. OPD-TV evaluates the same teacher under both views, measures how much enriched-view probability remains within a high-probability token set from the original view, and uses this compatibility to geometrically interpolate the teacher distributions. The student is trained using Jensen–Shannon divergence. On the TimeLens-corrected Charades, ActivityNet, and QVHighlights benchmarks, OPD-TV improves mIoU over direct privileged OPD by 0.7, 1.2, and 1.4 percentage points, respectively. Target ablations and additional diagnostics support the usefulness of regulating privileged supervision.

Critical strengths

Through targeted control measures, the study made a meaningful distinction between teacher accuracy and supervision transferability. In event-rich experiments, the paper fixed the student’s perspective, thereby distinguishing changes in teacher visual allocation from changes in model capacity. The comparisons of “off-target,” “labeling intervals,” and “expanded windows” in Table 2, combined with the non-monotonic student results in Figures 3–4, help concretize the core issue. These findings provide valuable insights into why stronger evidence of privilege does not necessarily lead to more effective supervision.

An interpretable target construction that yields consistent empirical improvements. Sections 3.1–3.3 and Appendix E demonstrate how geometric targets scale corrections for additional viewpoints while maintaining supervision from the same viewpoint. Table 3 compares the results of uniform, random shuffling, and binary shift strategies applied to the same training sequence, with targeted testing of token-level adjustments. Improvements were achieved compared to direct privilege distillation across all three benchmarks and all metrics reported in Table 1. The randomized controlled experiment in Appendix D.4 also quantifies the extent to which evidence-sensitive training contributes to the final results.

Critical weaknesses

Compared to the mathematical construction, the definitions of the achieved objectives are not sufficiently clear. Appendix D.2 notes that, prior to distillation, the teacher distribution is truncated to the first 100 tokens, while Appendix E assumes a positive distribution based on a general vocabulary. If truncation is applied separately to the two teacher views, their support sets may differ, thereby affecting log-probability normalization, geometric interpolation, and compatibility calculations. The paper does not specify the order of truncation, normalization, and interpolation, nor does it explain how zero probabilities are handled. Since support set preservation is a core mechanism, these choices may alter the essence of the method, not merely its numerical accuracy. This is an unresolved implementation issue and does not constitute evidence that the proof derivation itself is flawed.

The reliability of the incremental adjustment gain has not been fully quantified. Compared to Video OPD on the same view, most of the improvements stem from direct privileged supervision. OPD-TV achieves a 0.7–1.4 mIoU improvement over this baseline, and Table 3 shows a 0.7-point advantage over uniform transfer. These improvements are consistent, but uncertainty across runs is not reported. The process for selecting the uniform transfer coefficient is also not explained. Appendix D provides important view settings but does not include the optimized hyperparameters or value-mining thresholds shown symbolically in Figure 10. These omissions limit confidence in the method’s reproducibility.

Questions for authors

Is top-100 truncation applied separately to each teacher view or only after constructing the interpolated target? Please specify support alignment, probability normalization or flooring, and whether the reference set and compatibility are computed before or after truncation.

How was the uniform-transfer coefficient in Table 3 selected, and what evidence is available about variation across training runs for the direct, uniform, and adaptive targets under the same setup? Please also provide the optimization settings and numeric data-selection thresholds needed to reproduce this comparison.

3 comments
AuthorsOpenReview ↗

Author Response (W1, Q1): Support Alignment and Normalization

We thank Reviewer u737 for the careful technical reading and answer each question below.

W1 & Q1: Order of Top-k Truncation, Support Alignment, and Normalization

Support alignment. Top-k selection is performed separately for each distribution, before compatibility calculation and geometric interpolation. We form a shared support from the student's top-100 tokens, the top-256 tokens from each teacher view, and the sampled response token. Table 8 incorrectly lists the teacher setting as "Top 100". We will correct it to distinguish the student top-100 support from the teacher top-256 candidates used for reference-set construction. For every token in this union, we retrieve its original full-vocabulary-normalized probability under all three distributions. A token omitted from one view's top-k therefore retains its original probability under that view. Top-k omission does not assign zero probability or require probability flooring, and no separate top-k renormalization is applied. This alignment robustly handles non-overlapping top-k sets.

Reference-set construction. After support alignment, St(ρ)S_t(\rho) is selected exclusively from the same-view teacher's top-256 tokens as the smallest probability-ranked set reaching ρ=0.9\rho=0.9. The aggregate tail is excluded from reference-set selection. We verify that the same-view teacher’s top-256 candidates cover the required probability mass of 0.9. This coverage condition was satisfied throughout our experimental runs. The reference set therefore follows Eq. (6) directly. Compatibility is then computed as the enriched-view teacher's original probability mass on this reference set.

Target construction and normalization. After compatibility is computed, each distribution's probability mass outside the shared support is pooled into one shared tail coordinate. We perform geometric interpolation on the explicit tokens and the tail coordinate. We then normalize the target before computing symmetric JSD. JSD is evaluated exactly on this common compressed space. Appendix E applies to this finite space. Tail aggregation approximates full-vocabulary interpolation and JSD.

We will clarify the separate student and teacher top-k settings in Appendix D.2 and Table 8. We will also explain the tail approximation and coverage condition around Eq. (6) and in Appendix E. These exact implementation details are available for inspection in our anonymous code.

Takeaway: Our implementation preserves the reference-set definition and ensures proper normalization after top-k selection.

AuthorsOpenReview ↗

Author Response (W2, Q2): Stability and Reproducibility

W2 & Q2: Uniform-Coefficient Selection, Multi-Seed Stability, and Reproducibility Settings

  • Selection of uniform transfer weight. Mean compatibility of 0.933 corresponds to a mean adaptive weight of approximately 0.95. Assigning that weight uniformly also gives high weight to low-compatibility corrections. Such a target degenerates toward Direct Privileged OPD (r=1r=1). The tested constant weights r∈{0.4,0.6,0.8}r\in\lbrace 0.4,0.6,0.8\rbrace achieve 57.2, 57.4, and 57.3 mean mIoU, respectively, making r=0.6r=0.6 the best tested constant setting. The Shuffled transfer baseline in Table 3 provides the exact distribution-matched control: it preserves the numerical distribution of adaptive weights but permutes their assignments across tokens. Its drop to 56.8 mean mIoU, versus OPD-TV's 58.1, proves the importance of matching each weight to the corresponding token's local compatibility.
  • Multi-seed stability. We evaluate Direct Privileged OPD, uniform transfer, and OPD-TV across three random seeds (seeds 0, 1, and 2026) using the exact same ordered examples and update budget (Tab. R1).

(Note: The table below reports three-seed means and standard deviations, whereas Table 3 in the main paper reports a single representative run under the default hyperparameters. Both settings show consistent gains over the two baselines.)

Tab. R1: Three-Seed Stability Comparison (Mean over Charades, ActNet, QVH)

MethodMean mIoU (±\pm std)[email protected] (±\pm std)
Direct Privileged OPD56.8 ±\pm 0.543.2 ±\pm 0.5
Uniform transfer (r=0.6r=0.6)57.3 ±\pm 0.443.7 ±\pm 0.4
OPD-TV58.1 ±\pm 0.444.7 ±\pm 0.4

OPD-TV consistently outperforms both baselines. The mean paired gains over Direct Privileged OPD are +1.3 ±\pm 0.2 mIoU and +1.5 ±\pm 0.2 [email protected]. The gains over uniform transfer are +0.8 ±\pm 0.1 mIoU and +1.0 ±\pm 0.2 [email protected]. Paired bootstrap comparisons yield p<10−4p < 10^{-4} against both baselines, pooling test examples across all three benchmarks.

  • Optimization hyperparameters. The 8B student was fine-tuned using full-parameter training with AdamW, learning rate 2×10−62 \times 10^{-6}, weight decay 0.01, and BF16 precision. Student rollouts use multinomial sampling with temperature 1.0, consistent with y^∼πθ(⋅)\hat{y} \sim \pi_\theta(\cdot) in Eq. (10). Stochastic rollout sampling produces different training trajectories across seeds even with a fixed example order, accounting for the variation in Tab. R1. All target variants in Table 3 share these identical optimization hyperparameters.
  • Data selection. Tab. R2 lists these settings together with the data-selection thresholds. Fig. 10 and Appendix D.3 present the primary teacher-gain branch. The implementation also includes a stable auxiliary branch (ΔI≥−0.02\Delta I \ge -0.02) that preserves high-confidence grounding states. We will update Fig. 10 and Appendix D.3 to explicitly diagram both operational branches.

Tab. R2: Optimization Hyperparameters and Data-Selection Thresholds

ComponentParameterSetting
OptimizationTraining modeFull-parameter training (no LoRA)
OptimizationOptimizer / mixed precisionAdamW / BF16
OptimizationWeight decay0.01
OptimizationLearning rate / schedule2×10−62\times10^{-6}, 10 warmup updates, then constant
OptimizationGlobal batch / mini-batch16 / 16
OptimizationTraining budget1 epoch, 150 updates; evaluated at step 125
RolloutsDecoding / temperatureMultinomial sampling / 1.0 (1 rollout/ex)
Data selection (Fig. 10)1) Teacher-gain branchIK>0.30I_K > 0.30 and ΔI>0.05\Delta I > 0.05
Data selection (Fig. 10)2) Stable auxiliary branchIK>0.30I_K > 0.30, IU>0.30I_U > 0.30, and ΔI≥−0.02\Delta I \ge -0.02

Takeaway: The multi-seed and shuffled-weight controls support a consistent benefit from adaptive token weighting over the selected constant baseline.

With the discussion period closing tomorrow, we would greatly appreciate it if you could revisit our rebuttal and the revised manuscript. Please let us know if our response addresses your concerns, or if there are any remaining points we can clarify. Thank you very much for your time!

Reviewer_wPhuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how teacher views affect on-policy distillation for temporal video grounding, highlighting that richer teacher observations do not necessarily lead to better student performance. To address the information gap between teacher and student, the authors propose OPD-TV, which adaptively combines teacher predictions from the student’s view and a privileged view based on their compatibility. This approach aims to make privileged supervision more suitable for learning from the student’s limited observations. Experiments on three temporal video grounding benchmarks report consistent improvements over standard and direct privileged distillation baselines, with ablation studies examining teacher-view design and adaptive supervision.

Critical strengths
  1. Fixing the teacher model and student prefix while changing the teacher's visual input provides a clear way to study how the information state affects supervision. The fixed-budget event/context allocation experiment shows that better teacher predictions need not yield a better student. This identifies a concrete, testable issue arising from asymmetric observations in video OPD.

  2. OPD-TV improves all 12 dataset–metric combinations in the main table relative to Direct Privileged OPD. Table 3 compares constant weights, shuffled weights, and two hard-truncation variants on the same training sequence, providing supporting evidence for matching local weights to the corresponding tokens. Teacher configurations also include 4B, an 8B EMA teacher, and 32B, extending the evaluation beyond one external teacher.

  3. The appendices include a random training-pair selection control, JSD/KL comparisons, broader video-understanding evaluations, and training-time curves. They also explicitly report mixed outcomes in the low-compatibility group and acknowledge that compatibility does not identify every harmful correction. These disclosures help delimit the conclusions and distinguish aggregate improvement from local failures.

Critical weaknesses
  1. The score ctc_t measures agreement between two teacher distributions over a selected set. It does not directly use the student distribution, the event evidence actually visible to the student, or correction correctness. The same-view teacher's high-probability set can reflect its language priors and confidence rather than the student's visual support. Consequently, the algebraic decomposition in Equation (5) and KL monotonicity in Equation (14) do not by themselves establish that the method preserves correct, learnable corrections while suppressing unlearnable ones. In Appendix B.1, the enriched teacher's probability mass outside the set is exactly 1−ct1-c_t; plotting this quantity by compatibility group partly restates the definition rather than independently validating learnability or separating compatibility from confidence.

In Figure 7, the low-compatibility group, which receives the strongest regulation, declines by 0.8 mIoU and 1.6 [email protected] relative to direct privileged distillation. The authors report this candidly, but it calls for more cautious mechanistic claims in the abstract. Conditional analyses relating compatibility to student event coverage, correction correctness, teacher entropy/set size, and downstream benefit are needed, beyond showing that interpolation pulls the target toward the same-view teacher.

  1. The headline "+5.5/+6.2" points are total gains over Video OPD. The comparison that more directly isolates the regulation mechanism is the "+0.7/+1.2/+1.4" points over Direct Privileged OPD. In Table 3, the average mIoU advantage over constant weighting is 0.7 points. These are promising results, but the manuscript reports no repeated training seeds, standard deviations, or paired intervals, leaving their stability unclear. Sharing the training sequence in Table 3 is a valid control. Table 9 also evaluates OPD-TV on randomly selected examples, but does not provide the corresponding direct-versus-regulated distillation comparison on that random sample.

  2. In terms of novelty, the algebraic decomposition and geometric interpolation are relatively direct; the distinctive contribution lies in the compatibility score and its mapping to a transfer weight. This makes comparisons against simple alternatives, such as entropy/confidence- or JSD-based gating and arithmetic distribution mixtures, particularly relevant. The paper should also clarify its specific distinctions from cited approaches such as Fisher-projected OPD, decomposed OPD, and VAD.

  3. The fixed-budget experiment on the left of Figure 3 is informative, but increasing event frames also removes context frames. It therefore studies evidence reallocation rather than adding evidence while preserving all other information. More clearly, Table 6 shows that the experiment on the right of Figure 3 changes the student view from U64U_{64} to U128U_{128} and U256U_{256}. This also changes student information and rollout trajectories, so it cannot isolate an effect of additional teacher context on teaching quality. Furthermore, the teacher mIoU increase from intermediate to strongest enrichment in Figure 4 is only 52.1 to 52.2, without uncertainty estimates, which weakens a strong claim of continuously improving teacher accuracy. An experiment that fixes the student view while varying only teacher context would address this issue, as would separating the effects of adding event evidence and removing other context. The current results support non-monotonic student gains for particular view allocations, but the causal interpretation of a general mechanism should be correspondingly narrower.

Questions for authors
  1. After controlling for teacher entropy, reference-set size, token type, and student event coverage, does ctc_t still predict correction correctness or training benefit? How do you explain the simultaneous mIoU and [email protected] declines in the low-compatibility group in Figure 7?

  2. What are the multi-seed means, variances, and paired differences relative to Direct Privileged OPD and the best constant weight? Could you add at least one confidence/distribution-distance gating or arithmetic-mixture control, and report the search ranges for ρ\rho, rmin⁡r_{\min}, and the constant weight?

  3. Is top-100 truncation applied before or after compatibility calculation and interpolation? How are the two token sets aligned and omitted probabilities treated? How is normalizability ensured when independently selected supports do not overlap, and is JSD computed over the full vocabulary?

  4. Could you repeat the right-hand experiment in Figure 3 with the student fixed at U128U_{128}, varying only the teacher's context budget? Can adding event frames be separated from removing context frames? Does the teacher's 52.1-to-52.2 change in Figure 4 exceed evaluation and decoding variability?

  5. Can the teacher infer the annotated interval from timestamps or nonuniform sampling density? What happens when temporal structure is retained but images are masked or shuffled? Does regulation still outperform direct privileged distillation with noisy intervals, coarse windows, or automatically retrieved event frames?

7 comments
AuthorsOpenReview ↗

Author Response (W1, Q1): Compatibility and Transfer Benefit

We thank Reviewer wPhu for the careful reading and the concrete suggestions, which we address point by point below.

W1 & Q1: What Compatibility Measures and Why the Low Group Declines

Compatibility measures cross-view support shift and serves as a soft transfer signal. We evaluate correction correctness separately against the ground truth. OPD-TV therefore retains nonzero guidance through rmin⁡r_{\min} at low compatibility. Every training example contains event evidence under the student view, and 86.8% contain at least three in-event frames. Figure 2 (upper) illustrates the conceptual extreme of strict zero intersection to explain teleportation, whereas the observed training set has nonzero event coverage for every example. Mean boundary-token compatibility is 0.94 for examples with at least three event frames and 0.89 for those with fewer than three, linking lower compatibility to sparser student-visible evidence.

Outside mass is defined as 1−ct1 - c_t. Appendix B.1 visualizes this quantity across subsets, so it provides a descriptive visualization of the compatibility definition. Because mean outside mass in the Low group is 78.8%, we have cˉt≈0.21\bar{c}_t \approx 0.21, which strictly dictates our scaled transfer weight rˉt≈0.37\bar{r}_t \approx 0.37. We will make this descriptive role explicit in Appendix B.1.

Low-compatibility corrections can still provide useful guidance. On boundary tokens with ct<0.5c_t < 0.5, the enriched-view teacher moves closer to the ground truth than the same-view teacher in 83% of cases. Attenuation can therefore reduce the transfer of correct guidance. In the Low group of Fig. 7, OPD-TV trails Direct by 0.8 mIoU and 1.6 [email protected] but remains +1.4 mIoU above Same-view Video OPD. Removing all low-compatibility corrections in the Truncate-low ablation reduces mean mIoU from 58.1 to 55.5. Soft regulation retains some useful guidance in this group, and gains in the Mid (+1.3) and High (+0.2) groups improve aggregate grounding.

Conditional Analysis on Learnability. We test whether compatibility predicts transfer benefit beyond teacher confidence on the N=1,541N = 1{,}541 QVHighlights evaluation examples. We compute the partial Spearman correlation between boundary-token compatibility cmin⁡c_{\min} and per-example downstream gain (mIoUDirect−mIoUSame-view\mathrm{mIoU}^{\text{Direct}} - \mathrm{mIoU}^{\text{Same-view}}). The controls are same-view teacher entropy, reference-set size ∣St(ρ)∣|S_t(\rho)|, and in-event student frame count. Compatibility remains positively correlated with downstream gain (ρpartial=0.21\rho_{\text{partial}} = 0.21, p<10−4p < 10^{-4}). This relationship also holds in the top quartile of teacher confidence, defined by the lowest entropy (n=385n = 385, ρpartial=0.18\rho_{\text{partial}} = 0.18, p=3.7×10−4p = 3.7 \times 10^{-4}). These results support the value of ctc_t as a cross-view support measure beyond teacher confidence.

We will revise the abstract to state that OPD-TV "attenuates corrections with low cross-view support" instead of "suppressing corrections that are misaligned with the student's visual input." We will also explicitly discuss the mixed outcomes of the low-compatibility group in the experimental analysis section of the main text.

Tab. R1: Frame Coverage and mIoU across Compatibility Groups on QVHighlights (Fixed-128 Budget, Fig. 7 Groups)

GroupIn-Event FramesSame-viewDirect (vs. Same)OPD-TV (vs. Direct)
Low (cmin⁡<0.5c_{\min}<0.5)1.852.054.2 (+2.2)53.4 (−0.8)
Mid (0.5≤cmin⁡<0.90.5\le c_{\min}<0.9)3.558.061.5 (+3.5)62.8 (+1.3)
High (cmin⁡≥0.9c_{\min}\ge0.9)6.461.265.4 (+4.2)65.6 (+0.2)

Takeaway: Compatibility provides information about downstream transfer benefit beyond teacher confidence.

AuthorsOpenReview ↗

Author Response (W2, Q2), Part 1: Stability and Alternative Controls

W2 & Q2: Regulation-Specific Gain, Stability, and Alternative Controls

Table 3 isolates the gain from compatibility regulation. It fixes the student view, paired teacher views, training sequence, and optimization settings, and changes only how the privileged correction is transferred. Direct Privileged OPD uses the enriched-view teacher target without regulation, uniform transfer applies the same fixed weight r=0.6r=0.6 to every correction, and OPD-TV assigns the token-specific weight rt=rmin⁡+(1−rmin⁡)ctr_t=r_{\min}+(1-r_{\min})c_t. Under this matched design, OPD-TV improves over Direct Privileged OPD by 1.1 mIoU and 1.4 [email protected], and over uniform transfer by 0.7 mIoU and 0.9 [email protected].

  • Stability across seeds. To measure whether these regulation-specific gains are stable across runs, we repeat all three target constructions with seeds 0, 1, and 2026, using the same ordered training examples, teacher views, optimization settings, and update budget.

(Note: The table below reports three-seed means and standard deviations, whereas Table 3 in the main paper reports a single representative run under the default hyperparameters. Both settings show consistent gains over the two baselines.)

Tab. R2: Three-Seed Stability Comparison (Mean over Charades, ActNet, QVH)

MethodMean mIoU (±\pm std)[email protected] (±\pm std)
Direct Privileged OPD56.8 ±\pm 0.543.2 ±\pm 0.5
Uniform transfer (r=0.6r=0.6)57.3 ±\pm 0.443.7 ±\pm 0.4
OPD-TV58.1 ±\pm 0.444.7 ±\pm 0.4

OPD-TV consistently outperforms both baselines. The mean paired gains over Direct Privileged OPD are +1.3 ±\pm 0.2 mIoU and +1.5 ±\pm 0.2 [email protected]. The gains over uniform transfer are +0.8 ±\pm 0.1 mIoU and +1.0 ±\pm 0.2 [email protected]. Paired bootstrap comparisons yield p<10−4p < 10^{-4} against both baselines, pooling test examples across all three benchmarks.

  • Selection of uniform weights. To select the constant-weight baseline, we replace OPD-TV's token-specific rtr_t with a single weight shared by every correction and evaluate r∈{0.4,0.6,0.8}r\in\lbrace 0.4,0.6,0.8\rbrace under the same training setup. These settings achieve 57.2, 57.4, and 57.3 mean mIoU, respectively. We therefore use r=0.6r=0.6, the strongest tested constant weight and the midpoint of OPD-TV's transfer range [0.2,1][0.2,1].
AuthorsOpenReview ↗

Author Response (W2, Q2), Part 2: Data Selection and Sensitivity

W2 & Q2 (continued): Regulation-Specific Gain, Stability, and Alternative Controls

  • Alternative distribution-distance controls. KL divergence is asymmetric, but its scalar magnitude does not identify whether a change stays within the same-view support. Both sharpening among supported candidates and shifting probability toward unsupported candidates can produce a large KL divergence. Compatibility instead measures the enriched-view teacher's probability mass retained within the same-view support, distinguishing support retention from a shift beyond that support.

Keeping the training sequence and update budget fixed, we compare OPD-TV against two distribution-distance controls (Tab. R3). Both gates use the same minimum weight of 0.2.

  • KL-weighted transfer: Uses the forward KL divergence from the enriched teacher to the same-view teacher:

    gt=σ(τKL−DKL(q~t∥qt)).g_t = \sigma(\tau_{\mathrm{KL}} - D_{\mathrm{KL}}(\tilde q_t \parallel q_t)).

  • JSD-distance gate: Uses the symmetric JSD between the two teacher distributions:

    gt=max⁡(0,1−DJSD(qt∥q~t)τJSD).g_t = \max\left(0, 1 - \frac{D_{\mathrm{JSD}}(q_t \parallel \tilde q_t)}{\tau_{\mathrm{JSD}}}\right).

For both gates, the transfer weight is rt=0.2+0.8gtr_t = 0.2 + 0.8g_t. We report the best result for each control over its tested parameter settings.

Tab. R3: Distribution-Distance Controls: KL and JSD (Mean over Charades, ActNet, QVH)

MethodMean mIoU[email protected]
KL-weighted transfer57.343.7
JSD-distance gate57.544.0
Uniform transfer (r=0.6r=0.6)57.443.8
OPD-TV58.144.7

Both distribution-distance controls remain below OPD-TV's 58.1 mean mIoU, supporting the benefit of measuring cross-view support directly.

  • Data-selection control. To directly test whether the gain depends on evidence-sensitive data selection, we replace the evidence-sensitive training sequence with a duration-balanced random sample of 2,400 examples. Direct Privileged OPD and OPD-TV are trained on the exact same Random-2400 sequence with matched teacher views, optimization settings, and update budget.

Tab. R4: Random-2400 Data-Selection Control (Mean mIoU over Charades, ActNet, QVH)

MethodMean mIoU
Direct Privileged OPD56.4
OPD-TV57.5

OPD-TV preserves the same 1.1-point gain over Direct Privileged OPD under random training-pair selection, showing that the regulation gain does not depend on evidence-sensitive selection.

  • Hyperparameter sensitivity. We evaluate the sensitivity of the two compatibility hyperparameters while keeping the target construction, training data, and optimization settings fixed. We vary ρ∈{0.85,0.90,0.95}\rho\in\lbrace 0.85,0.90,0.95\rbrace with rmin⁡=0.2r_{\min}=0.2, and vary rmin⁡∈{0.1,0.2,0.3}r_{\min}\in\lbrace 0.1,0.2,0.3\rbrace with ρ=0.90\rho=0.90.

Tab. R5: Sensitivity to ρ\rho and rmin⁡r_{\min} (Mean over Charades, ActNet, QVH)

Settingρ=0.85\rho=0.85ρ=0.90\rho=0.90 (Default)ρ=0.95\rho=0.95rmin⁡=0.1r_{\min}=0.1rmin⁡=0.3r_{\min}=0.3
Mean mIoU / [email protected]57.9 / 44.458.1 / 44.757.9 / 44.557.8 / 44.358.0 / 44.5

Performance remains stable across the tested range and consistently exceeds Direct Privileged OPD and uniform transfer.

Takeaway: The repeated-run and alternative-control comparisons support a regulation gain beyond direct privileged supervision.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their comments. We have revised the paper to clarify the interpretation of compatibility, add the TimeLens comparison, explain the implementation more precisely, and include additional controls for the regulation gains. Changes are marked in red.

Main text. Table 1 now includes TimeLens-8B under the same inference protocol. The abstract, introduction, and Section 4.3 describe the observed teacher-accuracy plateau as enrichment increases. Section 4.4 discusses the mixed results in the low-compatibility group, where regulation can attenuate useful guidance. Figure 2's caption now distinguishes its illustrative zero-overlap case from the nonzero event coverage in our training examples and uses the correct panel labels. Section 4.4 also summarizes the repeated-run results, alternative transfer gates, random-sampling comparison, and student visual-evidence masking. Related Work adds the connection to privileged observations and the imitation gap.

Appendices. We expanded the explanations in the following sections:

  • A.2, teacher-view enrichment. We report the small change in teacher accuracy from intermediate to stronger enrichment and its uncertainty, alongside the decline in student performance.
  • B.1–B.2, compatibility analysis. We clarify that outside probability mass equals one minus compatibility by definition. We also report the gains and losses across compatibility groups, including the low group's decline relative to direct transfer.
  • D.1, TimeLens comparison. We specify the shared inference settings and compare the two models across the three benchmarks.
  • D.2–D.3, implementation and data selection. We describe how student and teacher token candidates form a shared support, how original probabilities are retained, and when compatibility, tail aggregation, interpolation, and normalization are computed. We also give the numeric thresholds for both training-pair selection branches.
  • D.4, random training-pair selection. We add Direct Privileged OPD on the same Random-2400 sequence. OPD-TV retains a 1.1-point mean mIoU gain under both selection methods.
  • E, target properties. We state which identities hold exactly on the compressed support and which operations approximate the full-vocabulary objective.

New Appendix F. F.1 reports three-seed results and paired gains, compares KL- and JSD-based transfer gates, and states how the configuration and checkpoint were selected. F.2 reports student event and background masking and varies the teacher view while holding the student view fixed. F.3 evaluates the same checkpoints with 128 uniformly sampled frames at inference. The complete results are in Tables 10–13.

Dear reviewers,

It seems that the authors have carefully responded to your reviews.

If you haven't already, please take your time to go over them and let authors know what you think, to your earliest convenience.

Best,
AC.

AuthorsOpenReview ↗

Core Insights and Rebuttal Summary

We thank the reviewers and the Area Chair for their constructive feedback and summarize our main findings and the experiments added during the rebuttal below.

Core insights and contributions

  1. A stronger teacher view does not automatically make a better teacher for the student. In temporal video grounding, giving the teacher denser frames from inside the annotated event improves the teacher's own predictions. Past a moderate level of enrichment, however, teacher accuracy plateaus while the distilled student degrades. When the teacher relies on dense visual evidence that the student cannot see in its sparser uniform view, the resulting supervision shifts probability toward boundaries that have little support under the student's input.

  2. Regulate privileged supervision by how well it is supported under the student's view. OPD-TV evaluates the same teacher under both the student's uniform view and the event-enriched view. It measures how much of the enriched teacher's probability mass remains within the high-probability candidate set of the same-view teacher, and scales the cross-view correction accordingly. This preserves boundary refinements that the student can ground in its own observations while softening unsupported jumps.

  3. Consistent grounding gains across benchmarks and teacher scales without SFT warm-up. Across Charades, ActivityNet, and QVHighlights, OPD-TV improves mean mIoU from 53.6 for Same-view Video OPD and 57.0 for Direct Privileged OPD to 58.1, matching TimeLens-8B (58.1) under the same inference protocol while using only 2,400 training examples and no SFT warm-up (Table 1). The same cross-view regulation also improves students distilled from a smaller 4B teacher, a matched 8B self-distillation teacher, and a larger 32B teacher (Table 5).

Key takeaways from the rebuttal

  • Visual reliance and timestamp controls (sN7f, wPhu). Masking the student's in-event frames on QVHighlights reduces OPD-TV from 67.6 to 35.1 mIoU, compared with 64.7 mIoU when masking an equal number of background frames, showing that the gains depend on visible event evidence (Appendix F.2). On the teacher side, replacing images with black frames while retaining the enriched timestamps drops teacher accuracy from 52.1 to 12.3 mIoU, confirming that supervision comes primarily from visual content rather than timestamp density.
  • What compatibility captures and the low-compatibility trade-off (J2Gv, SEwW, sN7f, wPhu). Compatibility increases with the event frames visible to the student and remains positively correlated with per-example transfer gain after controlling for teacher entropy, candidate-set size, and in-event frame count (Appendices B.1–B.2). In the low-compatibility group, where many enriched-teacher updates still move toward the ground truth, strong attenuation reduces some useful guidance: OPD-TV trails Direct Privileged OPD by 0.8 mIoU while remaining +1.4 mIoU above Same-view Video OPD. Soft regulation avoids the 2.6-point overall drop caused by discarding low-compatibility corrections altogether (Table 3, Section 4.4).
  • Multi-seed stability, alternative gates, and controlled budgets (J2Gv, SEwW, sN7f, u737, wPhu). Across three random seeds, OPD-TV (58.1 ± 0.4 mIoU) consistently outperforms uniform transfer (57.3 ± 0.4) and Direct Privileged OPD (56.8 ± 0.5), and surpasses KL- and JSD-based transfer gates under the same setup (Appendix F.1). The +1.1 mIoU gain over Direct Privileged OPD also holds on randomly sampled training data without evidence-sensitive mining (57.5 vs. 56.4, Appendix D.4). Fixing the student view at 128 frames while varying only the teacher view reproduces the non-monotonic direct-transfer curve and the benefit of OPD-TV (Appendix F.2), and the gains persist when inference uses the same 128-frame budget as training (56.0 vs. 54.8 mIoU, Appendix F.3).
  • Support alignment and training overhead (sN7f, u737, wPhu). We clarified how candidate tokens from the student and both teacher views are aligned using their original full-vocabulary probabilities before selecting the reference set and pooling the remaining mass into a shared tail coordinate, adding only 9% training time over Direct Privileged OPD (Appendices D.2, E).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.