acceptodds
Withdrawn from ICLR 2027

GLoR: Global-Local Ranking for Video Temporal Grounding

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Video temporal grounding (VTG) requires a model to return the temporal segment in an untrimmed video that corresponds to a natural-language query. In VTG, especially in long-video scenarios, candidate segments with lower relevance may receive higher confidence scores than target segments. Consequently, the model may fail not because the correct moment is absent, but because an inferior candidate is promoted to Top-1, turning an otherwise successful localization into an incorrect prediction. To address this issue, we propose a method named the Global-Local Ranking Grounder (GLoR), which decouples global candidate ranking from local boundary localization. GLoR adopts a global branch for candidate scoring and ranking and a local branch for boundary prediction. The global branch uses relevance evidence to determine the degree of match between a candidate segment and the query, while the local branch preserves high-frequency temporal variations and obtains more precise temporal boundaries by predicting probability distributions over start and end boundaries. In addition, IoU-based ranking and candidate-quality calibration further improve the accuracy of candidate scores. We evaluate GLoR on the TaCoS, Ego4D-NLQ, and Soccer-GMR datasets. Experimental results show that GLoR achieves state-of-the-art (SOTA) performance on TaCoS, Ego4D-NLQ, and Soccer-GMR, fully demonstrating its outstanding capability in candidate ranking and precise temporal localization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video temporal grounding, candidate ranking, video understanding,moment retrieval

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7EXwOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets candidate misranking in VTG, where a well-localized candidate is generated but outscored by a less relevant one. Built on FlashVTG, the proposed GLoR splits the shared temporal pyramid into a global ranking branch with GRF and a local boundary branch with LBD, and adds IoU-aware ranking and calibration losses on decoded candidates. Experiments on three datasets show consistent gains, mainly from better Top-1 selection.

Critical strengths
  1. The method is described in sufficient detail. The paper specifies the formulation of each component, the loss design, and the key hyperparameters, including bin settings, Top-K budgets, temperature, margins, and weight initialization and clipping.

  2. The method achieves consistent improvements on three datasets. The appendix further supports these results with multi-seed runs and paired bootstrap tests.

Critical weaknesses
  1. The novelty of this paper is limited, and its distinction from FlashVTG is not clearly stated. The base score in Eq. 5 is essentially FlashVTG's ASR under a different name, which the paper does not acknowledge. The remaining components largely transfer existing detection techniques: GRF corresponds to FCOS-style centerness, LBD to GFL-style distribution regression, and the candidate-level objectives closely follow IoU-aware quality ranking. Moreover, the claimed global–local decoupling is weak in practice, since both branches share the same pyramid and differ only by residual operators whose scale α\alpha is clipped to [0, 0.1].

  2. The comparison scope is outdated. Although the introduction motivates VTG through MLLM-based systems such as UniTime and VTimeCoT, the paper includes no MLLM-based method (e.g., Qwen3-VL and its VTG-tuned variants) or general-purpose multimodal embedding model (e.g., Qwen3-VL-Embedding, WeMM-Embedding) as a baseline, even though the evaluated videos fit well within their context budgets. Reporting these as reference rows, together with an accuracy–latency trade-off, would clarify where GLoR stands in the current landscape.

  3. The misranking problem addressed in this paper may largely stem from weak features. All experiments rely on SlowFast+CLIP or EgoVLP features, which may simply fail to separate visually similar candidates, and SDER itself measures similarity with CLIP features. It remains unclear whether the misranking problem, and thus GLoR's gain over FlashVTG, persists with stronger frozen representations such as InternVideo2 or Qwen3-VL-Embedding.

Questions for authors

Since GLoR is built on FlashVTG, could the authors report results on QVHighlights and Charades-STA, which are the main benchmarks used in the FlashVTG paper? This would allow a direct comparison with FlashVTG’s published results.

Reviewer_KeaVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses candidate misranking in video temporal grounding, where irrelevant video segments may receive higher confidence scores than the target segment. The authors propose the Global-Local Ranking Grounder (GLoR), which comprises a global branch for candidate scoring and ranking and a local branch for temporal boundary prediction. The method is evaluated on three datasets: TaCoS, Ego4D-NLQ, and Soccer-GMR.

Critical strengths

The paper is well structured and addresses an important problem in video temporal grounding: irrelevant candidates may receive higher confidence scores than the target segment. Such misranking can cause localization failure even when a correct candidate has already been generated, making improved candidate ranking practically valuable.

Critical weaknesses
  1. TaCoS and extended Soccer-GMR contain approximately five-minute videos, limiting evidence for longer temporal horizons. Although Ego4D-NLQ provides relevant validation, results on Ego4D-Goalstep, MAD, or different video-duration groups would strengthen these claims.

  2. Although Eq. (1) defines boundary decoding, reference-point generation, boundary prediction, scoring, and final selection are explained separately without a clear overall flow. Figure 2 omits key components such as PointGenerator. Eqs. (3)–(5) lack explicit indexing conventions and tensor dimensions, and several custom operators lack architectural details or precise references. Combining related equations—(6)–(7), (14)–(15), and (16)–(17)—would improve readability and reduce whitespace.

  3. Sensitivity analysis covers only some of the numerous hyperparameters. The 32 uniformly spaced boundary bins over [0.25,31][0.25,31] and fixed Gaussian bandwidth lack justification, particularly regarding temporal resolution, pyramid strides, and ground-truth distance distributions. Since these choices determine boundary coverage and target clipping, their rationale and robustness require clarification.

  4. BAM-DETR is a relevant omission given its boundary-focused modeling and localization-quality ranking. DeCafNet’s reported Ego4D-NLQ results are below its published results, without sufficient explanation of configuration and feature differences. HiTeA’s training-free setting also needs distinction from supervised methods. These gaps obscure whether gains arise from architecture, supervision, or implementation.

Questions for authors
  1. What duration range defines “long video”? Are duration-stratified results or evaluations on longer inputs available to show whether gains persist as the temporal search space expands?

  2. Could you clarify the complete inference pipeline and configurations of PointGenerator, global/local temporal operators, fusion gates, and prediction heads, including tensor dimensions and indexing in Eqs. (3)–(5)?

  3. Why use 32 boundary bins over [0.25,31][0.25,31] and a fixed Gaussian bandwidth? What fraction of training targets is clipped at each pyramid level, and how robust are results to these choices?

  4. Could you specify the training data, supervision, features, implementation changes, and checkpoint-selection criteria for all baselines? Unclear differences between supervised and training-free settings, together with unexplained deviations from published results, raise fairness concerns. How were comparable evaluation conditions and appropriate configurations ensured for each method?

Reviewer_3H2DOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GLoR, a dense video temporal grounding framework that explicitly separates candidate ranking from temporal boundary localization. It consists of a global branch that aggregates multi-scale temporal context and introduces Global Relevance Fields (GRF) to model event interior and center evidence, and a local branch that predicts start and end boundary distributions through Local Boundary Distributions (LBD). The model further incorporates IoU-aware ranking, candidate-quality calibration, and hard-negative supervision to align candidate confidence with localization quality. Experiments are conducted on TaCoS, Ego4D-NLQ, and a controlled five-minute Soccer-GMR benchmark. The reported results demonstrate improvements over several selected baselines, particularly in strict-IoU localization on TaCoS and Top-1 retrieval on Soccer-GMR.

Critical strengths
  1. The paper addresses a meaningful problem in video temporal grounding. Candidate misranking is a practical failure mode: a correct temporal segment may already be generated but fail to become the final prediction because its confidence score is poorly calibrated. Directly supervising candidate scores using the IoU of decoded intervals is a reasonable approach to this problem and provides a clear objective beyond conventional point-level classification and boundary regression.

  2. The paper provides a reasonably broad empirical evaluation and component-level ablations. Evaluations on TaCoS, Ego4D-NLQ, and Soccer-GMR cover different grounding scenarios, while the controlled Soccer-GMR protocol and ranking diagnostics attempt to examine candidate selection beyond conventional localization metrics. The reported improvements over selected baselines suggest that candidate-level supervision can be beneficial, although stronger and more consistent comparisons are needed to establish the generality of these gains.

Critical weaknesses
  1. The performance of existing baselines is inconsistent with their published results. The results reported for RGNet and DeCafNet on Ego4D-NLQ are substantially lower than those reported in their original papers and subsequent comparisons. These discrepancies raise concerns about the consistency of the experimental protocols and make the claimed improvements difficult to assess reliably. The authors should explain the differences and perform comparisons under consistent protocols. Without resolving these discrepancies, it is difficult to determine whether the reported improvements reflect the proposed method or differences in experimental settings.

  2. Several stronger baselines are missing from the experimental comparisons. In particular, SG-DETR [1] reports 56.71/44.70/29.90/40.90 on TaCoS for [email protected]/[email protected]/[email protected]/mIoU, respectively, outperforming GLoR's reported 54.86/43.36/28.74/39.39. Moreover, HieraMamba is explicitly discussed in the Related Work section but is not included in the experimental comparisons. Given that these methods are directly relevant to video temporal grounding and report stronger results on the evaluated benchmarks, omitting them weakens the paper's performance claims and prevents a convincing assessment of its competitiveness against recent state-of-the-art methods.

[1] Saliency-Guided DETR for Moment Retrieval and Highlight Detection.

  1. The ablation results do not substantiate the claimed benefit of LBD for precise boundary localization. The authors attribute the improvement in TaCoS [email protected] to distributional boundary prediction. However, Table 5 does not fully support this explanation. Adding LBD to the GLD+GRF configuration improves mIoU from 19.75 to 20.98, but decreases [email protected] from 15.99 to 14.46, a drop of 1.53%. This raises the question of whether LBD genuinely improves boundary estimation or primarily changes candidate-ranking behavior. The authors should provide a more controlled analysis that isolates boundary quality from ranking quality and compare LBD against conventional deterministic boundary regression to establish its independent contribution.

  2. The evaluation lacks experiments on MAD, an important long-video temporal grounding benchmark. The paper motivates GLoR partly by the difficulty of ranking candidates in long videos, yet evaluates it only on TaCoS, Ego4D-NLQ, and a controlled five-minute Soccer-GMR protocol. In particular, the Soccer-GMR videos are extended to approximately five minutes, which is insufficient to establish the method's effectiveness on substantially longer videos. MAD, which contains much longer videos, is commonly used to evaluate long-video grounding methods, including HieraMamba, RGNet, and DeCafNet. Omitting this benchmark leaves a significant gap in the empirical validation of the paper's long-video motivation.

Questions for authors
  1. Why do the reported RGNet and DeCafNet results on Ego4D-NLQ differ substantially from their published results? Can you clarify the experimental protocols and provide comparisons under consistent settings?

  2. Can you include comparisons with Saliency-Guided DETR and HieraMamba, and evaluate GLoR on MAD?

  3. Why does adding LBD decrease [email protected] in Table 5 despite the claimed improvement in precise boundary localization? Can you provide an analysis that isolates its contribution to boundary quality?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.