- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper addresses the difficulty of ranking highly overlapping candidates in MR, where truncated, overextended, and well-aligned intervals share similar content but differ in temporal extent. The proposed EventFieldNet augments a base span score with three explicitly supervised scoring fields: Evidence for interval-wide semantic relevance, Support for event coverage relative to temporal context, and Transition for directional boundary alignment, each trained with counterfactual contrasts built from moment annotations. Experiments on three datasets show improvements, mainly at strict IoU thresholds.
Critical strengths
The experimental section covers a wide range of analyses, including retrained ablations, field-removal studies across IoU thresholds and moment lengths, and a comparison between counterfactual and saliency-based supervision.
Critical weaknesses
-
The design of Support is overly complex relative to its contribution. It stacks several hand-crafted components, and its context correction relies on references selected from the model’s own provisional predictions, so errors in these predictions can propagate into the correction. However, removing Support at inference causes only a marginal drop, and the need to manually attenuate its ranking gradient suggests that its intended coverage role is not stably learned.
-
The evaluation scope is narrow, and the results only partially support the claimed advantage. The main paper evaluates only on QVHighlights, where the improvements over the strongest baselines are inconsistent across metrics, and the additional results on other datasets are mixed. Moreover, the paper includes no MLLM-based baselines or long-video benchmarks, and does not report computational cost, which is a concern given that the method scores all grid candidates and may scale poorly to longer videos.
-
The method relies on many hand-crafted designs and hyperparameters without corresponding ablations, including those in the Support construction, the ranking loss, and the score fusion. As a result, it is difficult to judge which design choices are essential and how robust the method is to these settings.
Questions for authors
Since the base path is a newly designed architecture rather than an existing detector, could the authors integrate the three fields into an established detector such as SG-DETR? This would help separate the contribution of the proposed fields from that of the base architecture, and would also support the claim that the approach generalizes to other temporal localization models.
2 comments
Response to Specific Questions
All section references below use the revised manuscript's numbering.
Q1: The current fields are trained jointly with EventFieldNet's candidate-scoring mechanism; direct integration into SG-DETR does not reproduce their gains.
EventFieldNet jointly ranks dense, highly overlapping candidates, with the field scores, base score, and Support reference correction learned within the same pipeline. SG-DETR uses decoder-generated candidates, different confidence semantics, and its native objectives. Directly adding the scores therefore need not provide suitable calibration; candidate geometry and optimization coupling are other possible factors.
Following your suggestion, we retain SG-DETR's original losses, attach E/S/T to its encoded clip features, and add their scores to the decoded confidence. Appendix F.3 reports test average mAP of 54.11 for native SG-DETR versus 50.68 after integration. This result does not support a direct plug-in gain. The proposed explanations remain factors to investigate, rather than established causes of the decrease.
Appendix F.3 reports this negative result, and Section 5 discusses the next step: a more efficient, loosely coupled field-scoring mechanism adapted to the candidate representations, confidence scales, and training objectives of established temporal localization frameworks.
Response to Main Concerns
We thank the reviewer for the careful assessment and for recognizing the breadth of our analyses. Your suggestions helped us explain Support more clearly and examine computational cost, cross-dataset performance, and parameter stability. The revision addresses these three concerns below; all references use its updated numbering.
W1: Support's complexity comes from assessing event coverage using both internal relationships and external context; its components serve distinct roles.
Highly overlapping candidates can share semantic content yet cover different parts of an event. Evidence scores clip-level relevance, and Transition scores directional boundary alignment. Support additionally needs to detect omitted event content, which requires relating the candidate to surrounding content as well as examining its interior.
S₀ captures interactions and persistence between adjacent clips, while reference regions and outer rings provide coverage and contextual information. Up to eight references accommodate multiple plausible locations; the candidate–reference IoU threshold of 0.3 excludes insufficiently overlapping references from the correction. Coverage counterfactuals require truncations to lower the score and coverage-preserving expansions to remain neutral. The ranking-gradient coefficient of 0.1 balances aggregate ranking and coverage supervision without changing the forward score or attenuating Support's own auxiliary loss. Sections 3.2–3.4 and Appendix A.2.2 explain these roles.
Table 5(a) tests reference correction and gradient attenuation separately: removing either reduces validation average mAP from 57.46 to 56.60/56.75. Table 18 tests the learned behavior: under InternVideo2/CLIP+SlowFast, Support's truncation-ranking accuracy increases from 31%/31% without its loss to 55%/52%, and its expansion absolute score gap decreases from 0.12/0.11 to 0.04/0.04. These results address both component utility and the coverage objective; the smaller fixed-weight inference-removal effect does not by itself characterize Support's role during training.
Reference errors are examined directly. Table 16 reports positive average Support contributions of 1.50 ± 0.16 / 1.10 ± 0.20 mAP points in the low-quality-reference group. Recomputing references after E/T removal adds only 0.025–0.069 points to the frozen-reference drops in Table 17. This supports the average contribution conclusions while retaining reference quality as a limitation.
W2: The revised presentation makes cross-dataset competitiveness clearer while explicitly reporting computational cost and long-video limitations.
Table 2 now brings the original Charades-STA and TACoS results into the main text. In the supplementary matched-split Charades comparison, EventFieldNet and SG-DETR have similar R1, while average mAP is 47.64 versus 42.95 (Table 8); each method retains its native recipe. Table 10 adds published MLLM comparisons, with their different inputs and fine-tuning or joint-training settings identified.
The current implementation incurs additional cost. Table 20 reports batch-1 inference latency of 25.30 ms for EventFieldNet versus 15.50 ms for SG-DETR on the same H20 GPU with FP32/no-TF32, starting from pre-extracted features and including post-processing.
Dense grid enumeration limits scaling because candidate counts grow quadratically: 2,850 candidates at 75 clips and 720,600 at 1,200 clips, as explained in Appendix F.2. Table 21 reports 17.03 GiB peak allocated memory at 1,200 clips, with 127.87 ms for the forward pass and 371.73 ms for post-processing. This is a scaling measurement, not a long-video accuracy benchmark. Appendix F.2 discusses sparse candidates and hierarchical processing as future directions, and the conclusion acknowledges the efficiency limitation.
W3: Mean performance is stable across the tested loss weights, Support gradient coefficients, and counterfactual margins.
We jointly scale the three field-loss weights by ×0.5/×2, change Support's ranking-gradient coefficient to 0.03/0.3, and scale counterfactual margins by ×0.5/×2. Across these six settings, test average mAP changes by −0.230 to +0.031 points relative to the same three-seed default, whose mean is 55.679. Table 5(b) summarizes these results; Table 19 reports validation/test metrics and paired changes, and Figure 5 shows variability.
Appendix E also reports learned-scalar, MLP, and alternative Transition-weight fusion results of 55.66/55.56/55.65, versus 55.68 for the same default. Together with the component ablations, these results distinguish the roles of the designs from sensitivity to their numerical settings. The stability conclusion is limited to the tested ranges and does not assume arbitrary settings or parameter interactions are harmless.