- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies temporal video grounding with multimodal LLMs trained by GRPO. When a video contains repeated or similar occurrences of the queried event, all sampled rollouts may pick a wrong occurrence and receive zero tIoU. These all-zero groups have zero group-relative advantage and provide nogradient. The authors call them dead groups and show that most dead-group rollouts overlap a verified distractor. The proposed DTP-OPSD adds an on-policy self-distillation term to GRPO. A teacher sharing the same weights sees a privileged video in which the expanded ground-truth window is marked with red frame borders and verified distractors with blue borders. The teacher scores the student's own prefixes, and a token-level Jensen–Shannon divergence pulls the student toward it. A three-level controller sets the per-group distillation weight to 0 when the teacher does not outperform the student, 2 for dead groups where the teacher does better, and 1 otherwise. The authors also build HI-TVG-300, a 300-example hard-interference set derived from TimeLens-100K. Starting from the TimeLens-SFT Qwen3-VL-8B checkpoint, one epoch on these examples improves over Pure GRPO by 0.50, 1.15, and 0.76 mIoU on the three TimeLens benchmarks, with seed standard deviations under 0.12. Supporting evidence includes ablations of the controller, privileged view, and data curation, a GRPO plus ground-truth SFT control, a compute-matched GRPO control, dead-group recovery statistics, and results at 2B and 4B scales.
Critical strengths
- The dead-group diagnosis is a precise and well-supported characterization of a real GRPO failure in TVG. Table 7 shows that 79.8 percent of dead-group rollouts overlap a verified distractor, which ties the problem to occurrence selection rather than recognition and motivates the method directly.
- The privileged teacher is parameter-free and input-only. Because teacher and student share weights and differ only in the video, the capability gap comes solely from the temporal cue, and nothing is added at inference.
- Distilling on student-generated prefixes keeps the supervision on-policy. The teacher evaluates the exact states the student visits, so the JSD signal does not vanish when all rollout rewards are zero, which is the case GRPO cannot handle.
- The distractor-aware design is validated rather than assumed. Table 3 shows that misplacing the red border hurts (64.44), that random blue borders slightly hurt (65.94), and that verified distractors help (66.71), so both cue types are shown to matter.
- The method is data-efficient. With 300 curated examples and 37 updates, DTP-OPSD matches or exceeds TimeLens-8B, which was trained with RL on 100K examples, and gains hold at 2B and 4B scales.
Critical weaknesses
- The dead-group problem has standard, cheaper fixes that are not compared. A temporal-distance shaped reward (used by Time-R1 and VideoChat-R1, and explicitly removed here) gives every rollout a nonzero gradient without any privileged view.
- The verified distractor annotations could be used directly as a penalty reward. The method already requires these intervals, so a negative reward for overlapping a blue window is the most direct use of the same information and is absent.
- Hint-injected or off-policy guided GRPO is not tested. Mixing the teacher's privileged rollouts into the student's group would revive dead groups using the same teacher and compute, without a distillation loss.
- The privileged distillation signal contributes little on its own. Fixed JSD gives 55.48, 54.02, and 65.78 mIoU versus Pure GRPO's 55.43, 53.62, and 65.95, so on QVHighlights the distillation alone is slightly worse than GRPO and the entire gain comes from the heuristic weighting.
- Figure 4 compares only to TimeLens-8B on OMTG-Bench and QV-M2. Without Pure GRPO on these benchmarks it is unclear whether the method itself helps under multi-moment annotations.
- The headline state-of-the-art claim is mostly inherited. Pure GRPO on 300 examples already matches TimeLens-8B (55.43 versus 55.28 on Charades), so most of the reported standing comes from the TimeLens-SFT initialization plus GRPO rather than from DTP-OPSD.
Questions for authors
- Why does JSD-only training collapse to 15.04 mIoU in 37 small updates? Is this a format collapse or genuinely wrong intervals?
- Given that Fixed JSD is no better than Pure GRPO on QVHighlights, how do you interpret the claim that privileged JSD "remains informative" where GRPO gives no signal?
- Which checkpoint performed the teachability filtering, TimeLens-SFT or base Qwen3-VL-8B? If TimeLens-SFT, the random-subset comparison in Table 4 partly measures hard-example mining on the training model rather than curation quality.
4 comments
Q3 (W3): Why not mix privileged teacher rollouts into the student’s GRPO group?
Answer: Thank you for this helpful suggestion. Guided rollouts are a reasonable alternative. Our choice of same-prefix distillation concerns where and how teacher supervision is applied.
Although teacher and student share weights, they generate responses under different inputs: the teacher sees the privileged video, while the student sees the clean video. Teacher responses therefore cannot simply be treated as on-policy samples from the clean-video student without accounting for their different sampling distribution.
Adding successful teacher responses can provide positive sequence-level examples and change the group-relative advantages of the student responses. DTP-OPSD instead keeps the original student-side GRPO groups and asks the teacher to evaluate the next-token distribution at each student-generated prefix. Supervision is therefore computed at the generation states actually visited by the student.
The independent teacher rollouts estimate teacher quality for the controller; the JSD targets are computed on the student’s own responses. This explains our design choice without assuming that guided rollouts cannot also be effective.
Q4 (W4 and Original Question 2): If Fixed JSD provides limited gains, what supports the role of privileged supervision and adaptive weighting?
Answer: Thank you for highlighting this distinction. We agree that the results do not support applying privileged JSD uniformly. The adaptive controller is an integral part of DTP-OPSD, and the existing controller ablation directly evaluates its two components.
| Method | Charades-TimeLens mIoU | ActivityNet-TimeLens mIoU | QVHighlights-TimeLens mIoU |
|---|---|---|---|
| Pure GRPO | 55.43 | 53.62 | 65.95 |
| Fixed JSD | 55.48 | 54.02 | 65.78 |
| Rescue only | 55.77 | 54.50 | 66.33 |
| Suppression only | 55.62 | 54.37 | 66.24 |
| Full controller | 55.93 | 54.77 | 66.71 |
Rescue only differs from Fixed JSD by doubling the JSD weight for all-zero student groups in which the teacher performs better; it never suppresses JSD. This change improves mIoU by 0.29, 0.48, and 0.55 points, respectively. Suppression only removes JSD when the teacher does not outperform the student, without applying the rescue multiplier. Combining both components gives the highest mIoU on all three benchmarks.
The fixed-subset analysis separately examines recovery by the full method. As also discussed in our response to Reviewer Y1As, Q2, we evaluate the same 142 examples that formed all-zero groups at initialization. An example is recovered when at least one of eight diagnostic completions has positive target tIoU.
| Method | Recovery of the initial 142 all-zero groups |
|---|---|
| Pure GRPO, one epoch | 38.7% |
| Pure GRPO, compute matched | 48.6% |
| DTP-OPSD, one epoch | 64.1% |
These analyses provide complementary evidence: the controlled ablation supports the contribution of rescue weighting to final benchmark performance, while the fixed-subset analysis shows that the full method recovers more initial all-zero groups, including compared with longer GRPO training under a matched compute budget.
By saying that privileged JSD can provide a signal when GRPO provides none, we mean that teacher–student distribution differences can yield token-level supervision even when group-relative advantages are zero. This does not guarantee that every teacher prediction is useful. The controller determines when to suppress, retain, or strengthen that supervision.
Q5 (W5): Does DTP-OPSD improve over the matched Pure GRPO baseline on OMTG-Bench and QV-M²?
Answer: Thank you for requesting this comparison. We evaluated the existing Pure GRPO checkpoint using the same single-interval, best-match protocol as Figure 4. DTP-OPSD improves over Pure GRPO by 0.58 mIoU on OMTG-Bench and 0.91 mIoU on QV-M².
| Method | OMTG-Bench mIoU | QV-M² mIoU |
|---|---|---|
| TimeLens-8B | 58.20 | 54.37 |
| Pure GRPO | 57.81 | 55.30 |
| DTP-OPSD | 58.39 | 56.21 |
All models predict one temporal interval, which is evaluated against its best-matching ground-truth interval. Results cover all 320 OMTG-Bench queries and all 323 QV-M² test queries.
TimeLens-8B is included as a reference model. The comparison between Pure GRPO and DTP-OPSD holds the TimeLens-SFT initialization and HI-TVG-300 training data fixed, directly measuring the contribution of our training method. These results concern identifying one valid occurrence under multi-moment annotations, rather than retrieving every annotated interval.
Q6 (W6): How much of the reported performance comes from TimeLens-SFT rather than DTP-OPSD?
Answer: Thank you for raising this attribution question. We acknowledge TimeLens-SFT as a strong foundation and assess our contribution through the matched Pure GRPO comparison: DTP-OPSD improves mIoU by 0.50, 1.15, and 0.76 points, with [email protected] gains of 1.28, 1.62, and 1.78 points.
For the Qwen3-VL-8B branch used here, the released TimeLens pipeline performs SFT before GRPO training. The model relationships are summarized below.
| Model | Starting checkpoint | Subsequent training |
|---|---|---|
| TimeLens-SFT | Qwen3-VL-8B | TimeLens SFT stage |
| TimeLens-8B | TimeLens-SFT | Original TimeLens GRPO training |
| Pure GRPO | TimeLens-SFT | GRPO on HI-TVG-300 |
| DTP-OPSD | TimeLens-SFT | GRPO with adaptive privileged JSD on HI-TVG-300 |
The last two rows share the same initialization, 300 training examples, and one-epoch schedule. Their difference measures the incremental benefit of DTP-OPSD.
We also examine occurrence correction, as discussed with Reviewer Y1As, Q3, and Reviewer XPsp, Q2. On a fixed set of 120 evaluation examples where the TimeLens-SFT prediction had zero target overlap and overlapped a verified distractor, Pure GRPO corrects 24/120 examples to tIoU≥0.7, whereas DTP-OPSD corrects 38/120, increasing the correction rate from 20.0% to 31.7%. This provides an example-level view of the improvement beyond the shared initialization.
Q7 (Original Question 1): Why does JSD-only training fall to 15.04 mIoU? Is this a formatting failure or inaccurate localization?
Answer: Thank you for this important question. Our diagnostic indicates that the degradation primarily reflects inaccurate localization: 99.1% of JSD-only outputs parse into valid intervals, but mIoU on those valid predictions is only 15.18.
We separate output validity from localization accuracy on QVHighlights-TimeLens using the existing evaluation parser:
| Method | Valid outputs | mIoU on valid outputs | mIoU on all examples |
|---|---|---|---|
| TimeLens-SFT | 99.6% | 57.57 | 57.34 |
| JSD-only | 99.1% | 15.18 | 15.04 |
| DTP-OPSD | 99.9% | 66.78 | 66.71 |
In the JSD-only ablation, we remove the GRPO loss while retaining student and teacher rollout rewards for controller estimation. JSD encourages agreement between the clean and privileged views, but agreement alone does not ensure correct temporal localization. Because both branches share the updated parameters, the teacher also changes during training. GRPO supplies the direct optimization objective based on target tIoU.
Related instability has been reported in arXiv:2605.18740, which observes collapse when using the current policy as teacher without teacher regularization, and arXiv:2607.01763, which reports forgetting and collapse in continual on-policy self-distillation. In our setting, retaining GRPO alongside privileged JSD yields 66.71 mIoU. The objective ablation and output diagnostic support using JSD as auxiliary supervision rather than as the standalone training objective.
Q8 (Original Question 3): Which checkpoint performs teachability filtering, and does the data ablation also measure hard-example selection?
Answer: Thank you for requesting this clarification. We use TimeLens-Qwen3-VL-8B-SFT, referred to as TimeLens-SFT throughout the paper, for teachability filtering. It is also the initialization for the subsequent training experiments.
The same checkpoint is evaluated with clean and privileged video inputs. We retain examples whose mean privileged-teacher tIoU exceeds 0.7 while the mean clean-student tIoU is below 0.3. The filter therefore includes model-dependent hard-example selection, together with a check that the privileged view provides useful guidance.
Accordingly, the change from 66.05 to 66.71 mIoU in Table 4 reflects adding this teachability filter after human verification within our curation pipeline. It should not be interpreted solely as an improvement in annotation quality. TimeLens also includes a model-based filtering stage before GRPO training; our additional criterion concerns the teacher–student performance gap under different video inputs.
The main method comparison then holds the selected data fixed: Pure GRPO and DTP-OPSD both start from TimeLens-SFT and train on exactly the same HI-TVG-300 examples. This measures their difference on a common curated training set.
Dear Reviewer 7FTP,
Thank you again for your constructive feedback on our submission. We wanted to follow up on the two reward-based alternatives you suggested. Q1 and Q2 of our rebuttal report GRPO with an IoU-plus-center-distance reward and GRPO with a distractor-overlap penalty.
Both baselines use the same initialization, training data, and training schedule as Pure GRPO. DTP-OPSD achieves higher mIoU than both on all three benchmarks.
We would appreciate your feedback on these additions and how they affect your assessment. Please let us know if you have any further questions.
Best regards,
The Authors
Thank you for the careful review and for recognizing the motivation, shared-weight teacher design, and supporting analyses. We address all six weaknesses and three questions below, combining the overlapping points in W4 and Question 2.
Q1 (W1): Could a temporal-distance reward address all-zero groups more simply?
Answer: Thank you for highlighting temporal-distance rewards as an important baseline. Such rewards can distinguish zero-IoU predictions when their temporal distances from the target differ. However, GRPO depends on reward differences within a group, rather than simply whether individual rewards are nonzero.
For example, consider a query asking for the second time a person picks up a cup. If all eight rollouts predict the same interval corresponding to the first occurrence, they have identical center distances from the target. Adding a distance term therefore gives all eight responses the same reward, leaving their group-relative advantages at zero. This example illustrates the condition required for distance shaping to restore a relative learning signal: it must introduce reward differences among the sampled responses. When the privileged teacher performs better, DTP-OPSD can instead supply additional token-level guidance on the student’s prefixes even when the student rewards are tied.
To examine the practical effect of distance shaping, we added a controlled baseline using the IoU-plus-center-distance reward defined in Appendix Eq. (9) of Time-R1. It uses the same TimeLens-SFT initialization, HI-TVG-300 data, one-epoch schedule, and rollout and evaluation settings as Pure GRPO; only the reward function changes.
The distance term leaves performance nearly unchanged on Charades-TimeLens and ActivityNet-TimeLens and improves QVHighlights-TimeLens mIoU by 0.18. DTP-OPSD exceeds this baseline by 0.53, 1.18, and 0.58 mIoU points, respectively. Thus, the example explains when distance shaping can provide a relative signal, while the matched training comparison shows that this reward modification alone does not reach DTP-OPSD’s final grounding accuracy in our setting.
We also clarify that VideoChat-R1 uses IoU plus format rewards for temporal grounding, without a temporal-distance term.
Q2 (W2): Why not use the verified distractor annotations directly as a penalty reward?
Answer: Thank you for suggesting this direct use of the annotations. We added a GRPO + distractor penalty baseline with the same initialization, training data, one-epoch schedule, and rollout and evaluation settings as Pure GRPO.
To protect valid target regions, we take the union of verified distractor intervals and remove its overlap with the ground-truth target. Denoting the remaining region by D−, we use
-\alpha\frac{|\hat{\tau}\cap D^{-}|}{|\hat{\tau}|}, \qquad \alpha=1.$$ Thus, overlap with the target itself is never penalized, even when target and distractor annotations overlap. | Method | Charades-TimeLens mIoU | ActivityNet-TimeLens mIoU | QVHighlights-TimeLens mIoU | | :--- | ---: | ---: | ---: | | Pure GRPO | 55.43 | 53.62 | 65.95 | | GRPO + distractor penalty | 55.49 | 53.58 | 65.24 | | DTP-OPSD | 55.93 | 54.77 | 66.71 | The penalty baseline performs similarly to Pure GRPO on Charades-TimeLens and ActivityNet-TimeLens, while its QVHighlights-TimeLens mIoU decreases by 0.71 points. DTP-OPSD exceeds this baseline by 0.44, 1.19, and 1.47 mIoU points, respectively. Both approaches retain the tIoU reward and use the same target and distractor annotations. The added penalty supplies sequence-level feedback for avoiding known distractor regions. DTP-OPSD uses those annotations to construct a privileged teacher view and provides next-token guidance on student-generated prefixes. Although the penalty baseline avoids teacher-side computation, this direct use of the annotations does not match DTP-OPSD’s grounding accuracy under the tested setting.