- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies on-policy distillation for temporal video grounding. During training, the teacher sees frames from inside the annotated event, while the student (Qwen3-VL-8B) sees 128 uniform frames. The authors find that more event frames make the teacher better but, past a point, the student worse. OPD-TV runs the teacher under both views and scales the enriched view's correction by a per-token compatibility score c_t. It improves mIoU over Video OPD by 1.8, 5.5 and 6.2 points on the TimeLens versions of Charades, ActivityNet and QVHighlights.
Critical strengths
-
An interesting observation about privileged teacher supervision. The paper identifies a meaningful supervision transfer gap: enriching the teacher's visual input can improve teacher accuracy without consistently improving student performance. The controlled experiments in Tables 2 and 4 help isolate the effect of privileged temporal evidence from teacher capacity. This is a useful finding, particularly because the required event annotations are already available in temporal grounding datasets.
-
A simple method with promising empirical results. OPD-TV introduces a straightforward compatibility-based regulation mechanism with a closed-form distillation target. The ablation experiments in Table 3 provide evidence for the benefits of adaptive regulation over simpler alternatives. The reported improvements across three benchmarks, together with the analysis of failure cases and the released code, strengthen the practical value of the work.
Critical weaknesses
-
Insufficient baseline comparisons and experimental reliability. Table 1 omits TimeLens-8B (Zhang et al., CVPR 2026), which uses the same Qwen3-VL-8B backbone with GRPO training and reportedly achieves an average mIoU of 58.0, nearly identical to OPD-TV's 58.1. Meanwhile, the paper's own GRPO baseline performs 3.6–5.4 points worse than TimeLens-8B, without sufficient details to explain the discrepancy. This raises concerns about the fairness of the comparisons. Furthermore, the additional improvement from compatibility regulation is relatively small (0.7–1.4 mIoU over direct privileged distillation and only 0.7 over constant weighting in Table 3). Since all results are based on single runs, their statistical reliability remains unclear. The apparent use of test-set performance for selecting configurations and checkpoints further complicates the interpretation of these gains. Together, these issues substantially weaken the empirical evidence supporting the method's superiority.
-
Limited novelty and insufficient validation of the proposed mechanism. The supervision transfer gap is closely related to the previously studied imitation gap in asymmetric learning (Weihs et al., 2021; Warrington et al., 2021), which limits the conceptual novelty of the central observation. More importantly, the paper does not directly establish that the proposed compatibility score identifies supervision that the student cannot effectively learn. For example, it does not demonstrate whether lower compatibility corresponds to student frames missing the target event, while the failure cases in Figures 5 and 7 suggest that regulation cannot reliably address incorrect enriched-teacher supervision. The claimed monotonic improvement in teacher quality is also supported by a very small difference at higher enrichment levels (52.1 versus 52.2 in Figure 4). Finally, the evaluation is restricted to a single student architecture family and temporal grounding task, leaving the broader applicability of the proposed mechanism uncertain.
Questions for authors
-
Baseline fairness: Could the authors include TimeLens-8B (Zhang et al., CVPR 2026) as a baseline and clarify the training configurations of all compared methods? In particular, why does the reported GRPO baseline substantially underperform TimeLens-8B despite using the same model backbone? A GRPO comparison trained on the same 2,400 examples would help establish whether the improvements can be attributed to the proposed distillation strategy.
-
Statistical significance and model selection: Could the authors report mean and standard deviation over at least three independent runs for Direct Privileged OPD, uniform-weight transfer, and OPD-TV? Given the relatively small improvements attributable to compatibility regulation, this is important for assessing their reliability. Please also clarify whether the teacher view, frame budget, and checkpoints were selected using validation data or test-set results, and provide a consistent checkpoint selection protocol.
-
Validation of the compatibility mechanism: What proportion of training examples have no overlap between the student's sampled frames and the annotated event interval? Does the proposed compatibility score systematically decrease in these cases? A quantitative analysis relating event coverage, compatibility scores, and downstream grounding performance would help substantiate the claimed mechanism.
-
Sensitivity to privileged-view construction: Do the K16 and K64 experiments reuse the training examples selected under the K32 configuration, or is the data selection performed independently for each setting? This distinction is important for interpreting the reported effects of teacher-view enrichment.
-
Robustness to inference frame budgets: The student is trained using 128 uniformly sampled frames but evaluated at 2 FPS, potentially involving up to 768 frames. Do the reported improvements persist when the inference frame budget is fixed at 128 frames? This comparison would help determine whether the gains depend on the mismatch between training and inference conditions.
6 comments
Author Response (W2, Q3): Compatibility and Transferability
W2 & Q3: Relation to the Imitation Gap and Validation of the Compatibility Mechanism
- Relation to the imitation gap. Weihs et al. [1] and Warrington et al. [2] study the imitation gap arising from privileged observations in reinforcement-learning and POMDP settings. OPD-TV addresses this imitation gap in the token probability simplex of an autoregressive LLM. Unlike RL approaches, it requires no reward signal or auxiliary policy. Instead, it directly regulates token-level distribution shifts using cross-view support. We will discuss these conceptual connections in the related work.
- Applicability to other tasks and architectures. The core formulation uses cross-view evaluation and structural target interpolation entirely within the token probability simplex. We evaluate it on temporal grounding with Qwen3-VL. The formulation makes no architecture-specific assumptions. It can therefore be applied to other settings with asymmetric visual observation, such as long-video QA and dense captioning.
- Event coverage. Counting the uniformly sampled student frames that fall inside each annotated event shows that every training example has nonzero event coverage and 86.8% contain at least three in-event frames. Figure 2 (upper) illustrates the conceptual extreme of strict zero intersection to explain teleportation, whereas the observed training set has nonzero event coverage for every example.
- Compatibility. Mean boundary-token compatibility reaches 0.94 on examples with at least three in-event frames and drops to 0.89 on examples with fewer than three, consistent with the 0.933 overall average in Fig. 4.
- Downstream performance. Lower-compatibility groups contain fewer in-event student frames. Tab. R3 shows that OPD-TV improves over Direct Privileged OPD in the Mid (+1.3) and High (+0.2) groups. In the Low group, strong attenuation limits the transfer of some correct guidance. OPD-TV trails Direct by 0.8 mIoU but remains +1.4 mIoU above Same-view Video OPD.
- Incorrect teacher corrections. Fig. 5 directly examines examples where the enriched-view teacher performs worse than the same-view teacher. Direct Privileged OPD incurs a 5.4-point mIoU drop, while OPD-TV reduces this drop to 2.3 points, mitigating more than half of the degradation. This shows that regulation reduces the damage from incorrect enriched-teacher supervision.
- Teacher performance in Fig. 4. The teacher's accuracy plateaus at extreme enrichment levels. The difference between 52.1 and 52.2 is within evaluation variance (). The teacher is 2.0 mIoU below . We will revise the manuscript to describe the teacher plateau and its implications for student transfer.
Tab. R3: Frame Coverage and mIoU across Compatibility Groups on QVHighlights (Fixed-128 Budget, Fig. 7 Groups)
| Group | In-Event Frames | Same-view | Direct (vs. Same) | OPD-TV (vs. Direct) |
|---|---|---|---|---|
| Low () | 1.8 | 52.0 | 54.2 (+2.2) | 53.4 (−0.8) |
| Mid () | 3.5 | 58.0 | 61.5 (+3.5) | 62.8 (+1.3) |
| High () | 6.4 | 61.2 | 65.4 (+4.2) | 65.6 (+0.2) |
Takeaway: Compatibility links student-visible event coverage to transfer quality and helps reduce damage from incorrect teacher corrections.
[1] L. Weihs et al., Bridging the Imitation Gap by Adaptive Insubordination, NeurIPS 2021.
[2] A. Warrington et al., Robust Asymmetric Learning in POMDPs, ICML 2021.
Author Response (Q4–Q5): Data Selection and Inference Budget
Q4: Whether the K16 and K64 Settings Reuse Examples Selected under K32
The and experiments in Fig. 3 reuse the 2,400 training examples mined under the default teacher view. Keeping the training videos and queries fixed allows Fig. 3 to isolate the effect of the teacher view on the same training sequence.
To test whether the peak depends on this shared selection, we evaluate two alternative training sets in Tab. R4. One setting mines 2,400 examples independently for each teacher view so that and no longer rely on examples selected by . The other trains all three teacher views on the same duration-balanced Random-2400 sequence so that no teacher view influences data selection.
Tab. R4: Teacher-View and Data-Selection Controls (Mean mIoU over Charades, ActNet, QVH)
| Teacher View | Shared | Independent | Random-2400 |
|---|---|---|---|
| 54.7 | 55.1 | 54.5 | |
| 57.0 | 57.0 | 56.4 | |
| 54.3 | 54.6 | 54.1 |
Takeaway: The preference for intermediate enrichment persists when examples are selected independently for each teacher view.
Q5: Robustness under a Fixed 128-Frame Inference Budget
We evaluate the same checkpoints using 128 uniformly sampled frames at inference to match the student's training view. Tab. R5 reports the results across all three benchmarks.
(Note: Tab. R5 uses 128 uniformly sampled frames for all methods, while Table 1 uses 2 FPS sampling with up to 768 frames. OPD-TV retains the highest mean mIoU under this fixed inference budget.)
Tab. R5: Fixed-128 Inference with Matched Training and Inference Frame Budgets (mIoU)
| Method | Charades | ActivityNet | QVHighlights | Mean mIoU |
|---|---|---|---|---|
| Same-view Video OPD | 50.1 | 47.2 | 58.9 | 52.1 |
| Direct Privileged OPD | 53.6 | 48.3 | 62.6 | 54.8 |
| OPD-TV | 53.6 | 51.2 | 63.1 | 56.0 |
Takeaway: The grounding gains persist when training and inference use the same frame budget.
Author Response (W1, Q1–Q2): Baselines and Statistical Reliability
We thank Reviewer J2Gv for the detailed review and for finding the supervision transfer gap a meaningful observation.
The GRPO baseline in Table 1 uses the same Qwen3-VL-8B-Instruct base model and the same 2,400-example training sequence as OPD-TV. TimeLens-8B starts from the same base model and performs cold-start SFT on 30K examples. It then uses this model to select harder examples from the 100K training pool for thinking-free GRPO with batch-normalized policy optimization. We evaluate the released TimeLens-8B checkpoint under our Table 1 inference protocol at 2 FPS with a 768-frame cap. We will explicitly add TimeLens-8B to Table 1 to ensure a comprehensive comparison. TimeLens-8B reports 58.0 mIoU, and our evaluation yields 58.1 mean mIoU, matching OPD-TV (58.1). TimeLens-8B scores higher on Charades, whereas OPD-TV scores higher on ActivityNet and QVHighlights. TimeLens-8B uses 30K SFT examples and a 100K pool for data mining. OPD-TV reaches the same mean performance with only 2,400 examples and no SFT warm-up.
Tab. R1: Comparison with GRPO and TimeLens-8B under the Table 1 Inference Protocol (mIoU)
Takeaway: OPD-TV matches TimeLens-8B under a common inference protocol.
(Note: The table below reports three-seed means and standard deviations, whereas Table 3 in the main paper reports a single representative run under the default hyperparameters. Both settings show consistent gains over the two baselines.)
Tab. R2: Three-Seed Stability Comparison (Mean over Charades, ActNet, QVH)
OPD-TV consistently outperforms both baselines. The mean paired gains over Direct Privileged OPD are +1.3 ± 0.2 mIoU and +1.5 ± 0.2 [email protected]. The gains over uniform transfer are +0.8 ± 0.1 mIoU and +1.0 ± 0.2 [email protected]. Paired bootstrap comparisons yield p<10−4 against both baselines, pooling test examples across all three benchmarks. The adaptive-regulation gain therefore remains consistent across runs.
Takeaway: The regulation gain persists across training seeds under a shared configuration and checkpoint-selection protocol.