EventFieldNet: Structured Event Scoring for Video Moment Retrieval

Highly overlapping candidates pose a ranking challenge in video moment retrieval: truncated, overextended, and well-aligned intervals can share query-relevant content yet differ in event coverage and excess context. We propose EventFieldNet, which makes semantic and temporal ranking criteria explicit through three complementary fields that augment learned base span scores. Evidence aggregates learned clip-level relevance across the entire candidate interval; Support assesses event persistence within the interval relative to local context derived from the model's own predictions; and Transition separately evaluates directional entry and exit at the two boundaries. The fields receive role-specific supervision; in particular, Evidence can be constrained by reciprocal counterfactual video–query pairing built from moment annotations instead of by clip-level saliency GT, with the same retrieval accuracy. With the same InternVideo2 features as the strongest baseline and no additional pretraining, EventFieldNet improves average mAP on QVHighlights by 1.7 points, with larger gains at stricter IoU thresholds.