- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VerifyGround, an interactive framework for video temporal grounding. The model first generates an initial temporal interval, verifies whether the event and its boundaries are supported by visual evidence, and then decides whether to relocate the event, refine the boundaries, or return the current prediction.
The authors construct GroundStep-30K for supervised fine-tuning and introduce action-specific process rewards for reinforcement learning. The method improves over TimeLens-8B on three TimeLens benchmarks and reports additional zero-shot results on TACoS.
Critical strengths
- The distinction between event relocation and boundary refinement is useful.
The paper makes a reasonable distinction between finding the wrong event and finding the correct event with inaccurate boundaries. These two errors require different observations, and the proposed framework addresses them with separate actions. This is a sensible design for interactive temporal grounding.
- The ablation studies provide useful evidence for the training design.
The experiments do more than compare the final model with existing methods. In Table 3, adding action-specific process rewards improves average mIoU from 57.8 to 60.6 over outcome-only GRPO. Table 4 also shows that the full interactive policy outperforms a one-pass baseline under a matched visual-token budget. These results support the value of intermediate supervision and adaptive observation.
Critical weaknesses
- The comparison with existing methods is incomplete.
My main concern is that TimeLens2 is missing from Table 1, despite being cited in the paper. The published results of TimeLens2-8B on Charades-TimeLens, ActivityNet-TimeLens, and QVHighlights-TimeLens are 58.6, 58.6, and 70.2 mIoU, respectively, all higher than VerifyGround's 56.4, 56.3, and 68.9. Even TimeLens2-4B reports higher mIoU on these benchmarks.
Although differences in inference settings may affect direct comparisons, omitting such a relevant baseline weakens the paper's claim of state-of-the-art performance. In addition, the TimeLens-8B results reported in this paper differ from those reported in the TimeLens2 paper. A controlled comparison under the same evaluation protocol is needed.
- The effects of privileged supervision and the computational cost are not sufficiently examined.
Appendix A states that after two unsuccessful Relocate attempts, the teacher is provided with the ground-truth interval to generate a recovery trajectory. Although this information is removed from the student-visible input, the paper does not report how frequently this procedure is used or how much it contributes to the final results. An ablation without these recovery trajectories would help clarify their impact.
The paper also matches visual-token budgets when comparing interactive and one-pass inference, which is useful. However, multiple reasoning turns and additional video observations introduce computational overhead beyond the visual-token count. Without reporting actual inference latency or GPU costs, it is difficult to assess the practical efficiency of the proposed approach.
Questions for authors
-
Why is TimeLens2 omitted from the main comparison despite being cited in the paper? Could the authors evaluate TimeLens2-4B and TimeLens2-8B under a comparable input and inference protocol, and clarify the differences in the reported TimeLens-8B results?
-
What fraction of GroundStep-30K trajectories involve ground-truth-assisted recovery? How does performance change if these trajectories are excluded from supervised fine-tuning?
-
Could the authors report average inference latency, GPU cost, and results across multiple random seeds? In particular, it would be useful to understand whether the gains from multi-turn evidence acquisition remain worthwhile when the additional computation is taken into account.
3 comments
Dear Reviewer o9pf,
We would like to thank you for your time and comments. We hope our previous response has adequately resolved your questions or concerns. As the deadline for the ICLR rebuttal period is approaching, we look forward to hearing your feedback on our response, and would be pleased to clarify any additional questions.
Best,
Authors
Dear Reviewer o9pf,
As the rebuttal deadline is approaching, we would like to briefly follow up on our response, which includes matched-budget comparisons with TimeLens2, an ablation of GT-assisted recovery trajectories, and inference latency, cost, and variability measurements.
Another reviewer has responded positively to the additional experimental evidence, and we would greatly value your perspective as well. If any concerns remain, we would appreciate the opportunity to clarify them before the deadline. If you find that the new evidence adequately addresses your concerns, we would be grateful if you could reconsider your assessment in light of these results.
Thank you again for your time and consideration.
Best regards, The Authors
We address the comparison, supervision, and efficiency concerns below.
We thank the reviewer for the careful assessment and for recognizing the distinction between event relocation and boundary refinement, as well as the ablation evidence supporting intermediate supervision and adaptive observation. We address the comparison, supervision, and efficiency concerns below.
1. TimeLens comparison and the 8B backbone.
Our Table 1 quotes TimeLens-8B's scores in TimeLens [1]; TimeLens2 reports different scores under its own evaluation configuration.
Table R1. Input configurations.
Our training data derive from TimeLens-100K. TimeLens2 also uses additional TimeLens2-93K and Ego4D-NLQ training data and up to 100K-token packed SFT sequences. We re-evaluated TimeLens2 on the same settings as ours. For VerifyGround, this covers initial input and all subsequent observations. We also evaluated VerifyGround trained from Qwen3-VL-8B and Qwen2.5-VL-7B.
Table R2. Matched-budget mIoU
VerifyGround outperform both TimeLens2 variants on most of four benchmarks.
2. Frequency and contribution of GT-assisted recovery.
GT-assisted recovery accounts for 22.7% of GroundStep-30K. Excluding these trajectories from SFT reduces average mIoU across the three benchmarks by 2.1 points, demonstrating their contribution to student performance.
We further examined teacher failures by restricting the video input to the GT interval with a 10-second margin on each side, removing distant distractors. The teacher achieves IoU > 0.5 on 74.3% of the evaluated samples. This suggests that long-context search and competing events can prevent the teacher from attending to relevant evidence. GT-assisted recovery redirects it to overlooked segments and encourages prediction correction, turning otherwise unsuccessful attempts into useful training trajectories. Remaining failures may involve wrong annotations or unsampled key frames. Together, the ablation support retaining these challenging examples through annotation-assisted recovery.
Table R3. SFT ablation and teacher diagnostic.
3. Serial inference latency, GPU cost, and random seeds.
We evaluated on Charades using NVIDIA H20 and SGLang with serial inference. SGLang's prefix/KV-cache reuse reduces repeated prefilling of shared history across turns. We repeated VerifyGround inference three times with different inference seeds. Three inference runs give:
Table R4. Inference-seed variability (mIoU).
Table R5. Serial latency and estimated inference cost.
The additional latency reflects multi-turn evidence acquisition and verification. For efficiency-sensitive use, the same model can return its first-turn prediction without verification, using 0.87× TimeLens's time with mIoU only 2.7 points lower. We retain verification by default because it checks proposed intervals against newly acquired visual evidence, enabling correction of unsupported event hypotheses and temporal boundaries. This step is designed to reduce hallucinated localizations and improve reliability. The two modes offer a flexible accuracy–cost trade-off.
We will incorporate these comparisons, recovery analyses, and serial-inference measurements into the revision.
[1] Zhang J, Wang T, Ge Y, et al. Timelens: Rethinking video temporal grounding with multimodal llms[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 10419-10429.