PAEM: Progression-Aware Event Matching for Spatio-Temporal Video Grounding

Spatio-temporal video grounding aims to localize the spatio-temporal tube corresponding to a natural-language query in an untrimmed video. Existing methods exploit query-related semantic cues to guide spatial and temporal localization. However, semantic relevance alone may be insufficient to determine the complete extent of a queried event, as salient local cues can lead to incomplete event spans, while broader context may still fail to distinguish candidates that share substantial content but differ in event structure. We argue that event progression, encompassing how an event develops over time and connects to surrounding activities, can provide a structural basis for distinguishing semantically similar candidates. In this paper, we propose Progression-Aware Event Matching (PAEM), which explicitly incorporates event progression into spatio-temporal matching. Specifically, PAEM combines stage-aware context matching and state-aware process matching to jointly capture contextual stage relations and internal event progression. To capture the stage relations, we conduct stage-aware context matching by aggregating query-relevant responses before, within, and after each candidate span. Meanwhile, state-aware process matching characterizes internal state dynamics through latent responses and conditional transitions. The two matching scores jointly determine the target entity and its temporal interval. Experiments across 4 vision-language backbones show m_vIoU improvements of 3.6%–9.9% ↑ over corresponding baselines on HCSTVG-v1/v2, demonstrating PAEM's effectiveness. When transferred to related grounding tasks in a zero-shot setting, PAEM yields consistent improvements over the corresponding baselines, with gains of up to 9.4% ↑ across the primary metrics, highlighting its generalizability.