- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces TimeLens2 that can predict single or multiple start and end timestamps across diverse video lengths, domains, query forms, and viewpoints. It proposes a pipeline that produces reliable temporal grounding supervision for multi-duration videos, and constructs TimeLens2-93K dataset. A temporal Wasserstein reward is proposed to complement tIoU by providing distance-aware feedback for non-overlapping predictions.
Critical strengths
-
This paper proposes a pipeline for constructing annotations for the temporal grounding task and demonstrates the effectiveness of each stage of the pipeline through the experiments presented in Table 3.
-
This paper propose a new temporal grounding reward and demonstrates the effectiveness across various models and benchmarks.
Critical weaknesses
- Dataset construction is a key contribution of this paper, but the annotation pipeline relies on the initial coarse proposals and may miss relevant intervals in multi-span tasks.
Questions for authors
-
Why do the authors use Qwen3-VL-30B-A3B for local grounding rather than Qwen3-VL-235B-A22B, which is already used for caption generation and boundary refinement?
-
The temporal Wasserstein reward favors predictions closer to the ground truth even when they have zero overlap. While Table 10 and Table 11 support its overall usefulness, could this preference be misleading when nearby intervals contain unrelated actions and similar events occur farther away? Could the authors analyze such cases and compare with alternative reward designs, such as the timestamp-aware IoU reward in Time-R1?
-
Since both grounders operate on the selected coarse proposals, they may agree on the retrieved intervals while missing relevant events outside those proposals. Subsequent verification and refinement focus on the selected evidence. Is there a full-video verification step to detect omitted occurrences, particularly for multi-span queries?
3 comments
Q1: Why do the authors use Qwen3-VL-30B-A3B for local grounding rather than Qwen3-VL-235B-A22B
The main reason is compute cost under our limited budget:
- Too many queries to annotate with the 235B model. Kimi-K2.5 generates a large number of candidate queries, which yield over 1M query–clip pairs to be grounded. Running Qwen3-VL-235B-A22B on every pair would cost roughly more compute for this stage, which exceeds our compute budget. Qwen3-VL-30B-A3B activates only 3B parameters (vs. 22B), so it can annotate all candidates at a much lower cost.
- 30B-A3B is nearly as accurate. As shown in Table 1, Qwen3-VL-30B-A3B performs close to Qwen3-VL-235B-A22B on short-video grounding, which matches our setting since local grounding runs on short 20–60s clips. It therefore offers a much better accuracy–cost trade-off.
Q2: Could 's preference for closer predictions be misleading when nearby intervals contain unrelated actions and similar events occur farther away? Comparison with Time-R1 [1].
Thank you for raising this question. Below, we explain how 's preference for closer predictions can help boundary refinement despite this risk, and provide a controlled comparison against Time-R1 [1].
1. Why preferring closer predictions can help boundary refinement. As a geometric signal, itself does not assess semantic relevance, so a nearby unrelated interval can in principle score higher than a distant one. Nevertheless, the following mechanisms and evidence support its practical benefit:
- Sharp distance decay and unnormalized GRPO advantages: For a predicted and target interval of length separated by gap , and , which is sizable only near the target boundary (0.37–0.14 for ) and vanishes quickly beyond it (0.05 at , 0.018 at , beyond ). Crucially, our GRPO advantages are mean-centered without dividing by group standard deviation (Table 12 in the revised manuscript; Table 11 in the original submission), so tiny reward differences between non-adjacent unrelated intervals and distant distractors are not amplified.
- Query-conditioned rollouts support boundary refinement: Rollouts come from a visually grounded MLLM conditioned on the query, which provides semantic guidance alongside the geometric reward. Nearby zero-overlap rollouts () can land on the target event's immediate setup or aftermath, whereas distant rollouts may select similar false-positive distractors. In such all-zero-tIoU groups, favoring the boundary-adjacent candidate can pull near-miss boundaries into the target.
- Empirical evidence (Table 11 in the revised manuscript; Table 10 in the original submission): Across all 4,332 zero-overlap predictions from tIoU-only RL, adding recovers 21.9% of near misses ( target length) and 15.8% of mid-distance misses () into positive overlap with the ground truth (7.5% and 4.0% mIoU), vs. only 5.7% for far misses (, 66.5% of cases), supporting the benefit of this distance signal for recovering near-miss predictions.
2. Controlled comparison with Time-R1's timestamp-aware IoU [1].
To compare reward designs in a complementary direct-RL setting, we ran GRPO on Qwen3.5-4B (no SFT):
| Reward | Charades | ActivityNet | QVHighlights | VUE-TR | VUE-TR-V2 | Avg. |
|---|---|---|---|---|---|---|
| No RL | 47.9 | 53.5 | 66.9 | 35.4 | 27.7 | 46.3 |
| tIoU | 55.1 | 57.7 | 68.4 | 51.0 | 45.0 | 55.4 |
| NGIoU (MUSEG [2]) | 50.8 | 57.6 | 67.6 | 52.6 | 45.0 | 54.7 |
| Timestamp-aware IoU (Time-R1 [1]) | 54.3 | 55.9 | 64.2 | 48.9 | 46.3 | 53.9 |
| tIoU + Temporal Wasserstein (ours) | 55.6 | 59.1 | 70.1 | 53.2 | 47.5 | 57.1 |
NGIoU already includes tIoU. Time-R1 [1] multiplies single-interval tIoU by a boundary-offset penalty; because it is multiplicative in tIoU, it is still zero for all disjoint predictions and even underperforms plain tIoU. In contrast, our reward achieves the best result on all five benchmarks, with the largest gains on multi-interval VUE-TR (+2.2) and VUE-TR-V2 (+2.5). We have added this comparison to Appendix A.4.
References
[1] Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding. NeurIPS 2025.
[2] MUSEG: Reinforcing Video Temporal Understanding via Timestamp-Aware Multi-Segment Grounding. ACL 2026.
A Brief Follow-up
Dear Reviewer M2yS,
Thank you again for the time and effort you put into reviewing our work. Your questions have helped us improve the paper’s analysis of annotation completeness and reward behavior.
As the discussion period ends in less than 48 hours, we would be grateful for your thoughts on our responses and additional results. Please feel free to raise any remaining concerns; we would be happy to provide further clarification.
All the best,
The TimeLens2 Authors
W1 & Q3: The annotation pipeline may miss relevant intervals in multi-span tasks
Thank you for your comment. Our pipeline aims to recover all relevant occurrences through full-video caption indexing and global query–proposal generation. Below, we explain these efforts to improve full-video recall, their remaining limitations, and the human audit quantifying omissions.
1. How full-video recall is handled. To minimize missed intervals, our global-to-local pipeline enforces full-video coverage before local grounding. Every video is first partitioned into contiguous, non-overlapping scene clips spanning 100% of the timeline, with
Qwen3-VL-235B-A22Bgenerating a detailed dense caption for every clip. Rather than writing queries independently and searching post hoc,Kimi-K2.5reads the complete video caption hierarchy to jointly synthesize each query and select its supporting candidate clips, under explicit instructions to collect all occurrences across the video; within each selected clip, both local visual grounders can further output multiple intervals whenever an event recurs.Nevertheless, we admit these mechanisms do not guarantee exhaustive recall: an occurrence may still be omitted by the captioner or missed by a local grounder. Eliminating such omissions via brute-force verification offers a poor cost–quality trade-off, e.g., running both visual grounders on every clip across hour-long videos, would dramatically inflate the already 2M+ grounding calls while introducing false positives from unrelated distractors. We therefore conduct a full-video human inspection to directly evaluate and quantify the remaining omissions.
2. Quantifying omitted occurrences via human audit. We conducted a human audit on a subset of 1,000 queries (500 single-span + 500 multi-span) sampled from videos across all five duration bins (<2 min to ≥20 min). For each query, human annotators watched the entire unclipped video from start to finish to verify query validity, semantic precision, boundary accuracy, and any omitted occurrences outside the proposed clips:
Across the 500 multi-span queries, full-video inspection yields 94% query-level span recall and a 90% complete-set rate. Even in the longest duration bin (≥20 min), where proposal omissions are most challenging, recall remains 97.4% with an 89.5% complete-set rate. These results quantify the residual omissions in the audited subset and support the practical effectiveness of full-video indexing followed by local grounding, while keeping annotation tractable at scale.