- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the limitations of tIoU for both short and long events in video temporal grounding. The authors propose a duration-aware evaluation and sample selection strategy, and combines dIoU with boundary rewards for RLVR training. Experiments show the superior performance on several VTG benchmarks.
Critical strengths
-
The paper provides a clear analysis of the different effects of tIoU on short and long events. The motivation is clear and reasonable.
-
The method achieves superior performance on several benchmarks. The paper also provides extensive ablation studies on different reward functions and sampling strategies.
Critical weaknesses
-
As shown in Table 3, the new filtered dataset brings considerable performance gains, especially for long events. For a fair comparison, the authors should report the results of the full method on both the original TimeLens-100K and the cleaned BiVTG-100K datasets under the same training settings.
-
The paper finds that different rewards work better for short and long events. However, the final reward simply combines dIoU and boundary rewards with fixed weights, rather than introducing a more principled duration-aware design. This also introduces additional hyperparameters. Moreover, the combined reward does not consistently outperform individual rewards in Table 3. The novelty and motivation of this reward design are therefore not sufficiently strong.
Questions for authors
In Weakness.
Response to Reviewer zALy
Thank you for your careful review and comments. Our responses are detailed below.
Q1: How much of the improvement comes from the training algorithm when the data are held fixed?
First, we have used BiVTG-100K for this comparison because hard-sample selection on the original TimeLens-100K can misclassify audio-dependent queries as difficult visual grounding examples before RLVR.
Table R.1 directly compares the baseline configuration with our method. Both use the same BiVTG-100K SFT checkpoint and the same training data, so the comparison isolates the joint contribution of sample strategy and reward design.
With the training data held fixed, our method improves short-event grounding on all three benchmarks, consistent with its aim of mitigating the short-event degradation observed under tIoU-only RLVR. Long-event performance improves on ActivityNet and decreases on Charades and QVHighlights. Rows 1–3 of Table 3 show the separate contribution of data refinement, which improves long-event grounding at the SFT stage.
Together, the refined dataset, mixed reward, and sampling strategy improve over the prior state of the art on both short and long events across all three benchmarks.
Q2: What motivates the mixed reward, and where does the novelty lie?
The motivation for the mixed reward comes from the reward hacking behavior we observe under tIoU-based RLVR. The mixed reward itself is not complex for avoiding more hyper-parameters beyond weights. Our novelty claim centers on two underexplored observations and the following duration-aware framework they motivate.
1. Evaluation bias of tIoU for short events. As Figure 1 shows, a small absolute boundary shift can cause a large drop in tIoU for a short event, even when the predicted timestamp is close to the annotated one. We then correct the conventional evaluation by including boundary metrics that provide graded feedback on temporal error.
2. Interval inflation under tIoU-based RLVR. As Figure 2 shows, optimizing tIoU can encourage the model to predict excessively long intervals. Table 3 shows the associated trade-off. Training with tIoU alone improves long-event localization but can degrade short-event grounding. The mixed reward addresses this behavior by combining overlap-based supervision from dIoU with a boundary term that penalizes predictions that extend beyond the annotated interval, which is the motivation for our mixed reward.
The single-reward ablations in Table 3 show why the mixed reward does not consistently outperform each individual reward. Overlap-based supervision favors long events, while boundary supervision is particularly effective for pinpointing short events. The fixed mixture balances these objectives across the training set for this trade-off.
Thank you again for your comments and valuable questions.