acceptodds
Under review as a conference paper at ICLR 2027

STEER: State-Transition Event Evidence Reasoning for Training-Free Video Temporal Grounding

Abstract

Video Temporal Grounding (VTG) aims to localize the temporal boundaries of a target event in an untrimmed video according to a natural language query. Exist- ing training-free methods typically identify target moments based on video-query semantic relevance. However, semantic relevance does not necessarily indicate that the target event is actually occurring. Moments before and after the event may contain similar persons, objects, and scenes, leading to ambiguous temporal boundaries. To address this issue, we propose STEER (State-Transition Event Evidence Reasoning), a training-free framework that explicitly reasons event pro- gression through complementary semantic, motion, and query-independent cap- tion evidence. STEER first parses the query into a structured event attributes, which guides the construction of complementary evidence from video-text seman- tics, action-aware human motion, and query-independent video captions. These cues are jointly used for event-state reasoning, and the resulting state transitions are used to predict the temporal boundaries. Experiments on Charades-STA and ActivityNet Captions demonstrate competitive performance, while ablation stud- ies further validate the effectiveness of the proposed evidence construction and state-transition reasoning. Code and related resources will be publicly available upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Temporal Grounding, Training-Free Learning, Event Progression, Event-State Reasoning, Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mSFQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

STEER is a training-free method for video temporal grounding. It combines VideoCLIP similarity, action-aware motion cues, and query-independent captions to help a frozen VLM identify whether an event has started, is ongoing, or has ended. It then uses event-support scores to predict temporal boundaries. Experiments on Charades-STA and ActivityNet Captions show competitive results, with ablations examining the evidence sources and localization strategy.

Critical strengths

The paper presents a clear idea: semantic similarity alone may not tell us whether an event is actually happening. Combining it with motion cues and query-independent captions is a reasonable approach to this problem. The overall pipeline is easy to follow, and the ablation results suggest that the additional evidence, particularly action-aware motion, helps improve temporal localization.

Critical weaknesses
  1. The core localization procedure is not fully specified. The paper does not explain how event-support scores are computed or give clear boundary-decoding rules. The ablations also do not isolate the benefit of state reasoning from that of the additional evidence.

  2. The contribution over prior work is unclear. DART already uses temporal stage labels for grounding, but is omitted from the main comparison. A clearer distinction and a comparison under matched settings are needed.

Questions for authors
  1. What formula converts the VLM outputs and evidence into continuous event-support scores? What thresholds and stopping rules are used for boundary decoding?

  2. does the VLM receive neighboring frames or previous state predictions? If it only sees the current frame, how does it distinguish BEFORE from AFTER when the observations look similar?

  3. how does STEER’s state reasoning differ from DART’s temporal role annotation, and why is DART omitted from Table 1?

Reviewer_pHdNOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes STEER (State-Transition Event Evidence Reasoning), an event-state reasoning framework for training-free VTG. The core idea is that semantic relevance alone is insufficient to determine whether the target event is actually occurring, because moments before, during, and after an event may share similar visual content. STEER decomposes the query into structured event attributes, then constructs three complementary evidence sources: video-text semantic similarity via VideoCLIP, human motion evidence via action-aware pose keypoint displacement, and query-independent video captions generated by a frozen VLM. Experiments are conducted on Charades-STA and ActivityNet Captions under fully training-free settings, with competitive zero-shot results reported against prior methods.

Critical strengths
  1. The problem motivation is clear and well-illustrated. The paper correctly identifies that semantic similarity scores tend to be high across multiple stages of the same event (before, during, after), leading to ambiguous temporal boundaries. Figure 1 effectively contrasts existing similarity-based paradigms with the proposed event-state reasoning approach.

  2. The action-aware pose selection is a thoughtful design.

Critical weaknesses
  1. The main contribution appears to be the integration of existing Video-Text Semantic Evidence, Human Motion Evidence and Query-Independent Video Captioning with event reasoning, rather than a fundamentally new modeling framework.

  2. The method heavily relies on the accuracy of pre-trained VLMs. If these external modules fail in specific scenarios, it directly impacts overall localization performance. Furthermore, the experimental section lacks quantitative analysis of these external module failures.

  3. Inference efficiency is not quantitatively evaluated. Appendix A.4 provides a qualitative breakdown (offline vs. online components) but reports no actual FPS.

  4. Only two datasets are evaluated. The experiment results are not sufficient.

  5. Evidence has been widely used for video understanding [1,2], thus, the novelty of the whole paper is weak. [1]Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence, cvpr 2026 [2] Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority, icml 2026

Questions for authors

More large scale datasets should be used, such as TACoS, QVHighlights, and EventNet.

Reviewer_GdwzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes STEER, a training-free method for video temporal grounding. First, a frozen VLM (Qwen2.5-VL-7B) parses the query into a subject, an action, objects, a relation and an action type. Then the method collects three types of evidence for each sampled frame: VideoCLIP similarity, the motion of action-related body joints from a pose estimator, and a caption that the VLM writes without the query. The VLM uses this evidence to classify each frame as BEFORE, DURING, AFTER or UNCLEAR, and the method gives each frame a support score. A peak search and a boundary scan on the support scores then give the start and end times.

Critical strengths

None.

Critical weaknesses
  1. The gain over the backbone is small. Qwen2.5-VL-7B can predict the boundaries directly, and its reported mIoU on Charades-STA is only about 2 points lower than that of STEER (arXiv:2502.13923, Table 8). But STEER needs many more VLM calls, and the paper reports no runtime.
  2. Important comparisons are missing. The paper cites TFVTG (ECCV 2024) and HiTeA (ICLR 2026) but does not compare with them, and TFVTG is better than STEER on some metrics. Table 1 also omits strong recent models with zero-shot transfer, such as Time-R1 and TimeLens, which uses the same backbone and has much better performance.
  3. The method does not use state transitions as the paper claims. The VLM classifies each frame independently, and no step enforces the order BEFORE → DURING → AFTER. The boundaries come from peaks in a support score, not from the state transitions.
  4. Important steps are not defined. The paper gives no formula for the support score, no rule to compare candidate peaks, and no thresholds for the boundary scan. Thus, other researchers cannot reproduce the results.
  5. No long video VTG and TimeLens-Bench results. Ego4d-NLQ, MomentSeeker, and VUE-v1/2 are commonly used long video temporal grounding benchmarks and no results are reported for them. Also, TimeLens-bench (https://arxiv.org/abs/2512.14698) provides revised annotations for Charades-STA and ActivityNet, and no results were reported for them either.
  6. The novelty is limited. TFVTG already models the transition into an event and decomposes the query with an LLM. VTG-GPT already uses frame captions. The new parts, pose evidence and a per-frame state label, are no more than additional engineering effort and give small gains.
Questions for authors

I have no additional questions beyond those listed in the weaknesses part.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.