- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper illustrates a new method, Sentence-to-Paragraph Grounding (S2PG) which is a lightweight model reformulating Video Paragraph Grounding (VPG) as a sequence-level candidate selection problem and improves the performance.
Critical strengths
The method is easy and effective on the dataset present in the paper.
Critical weaknesses
The test sets are limited and more experiments should be done on more datasets for solid experiments and conclusion.
Questions for authors
How does this paper inspire related works and further study? Could you please provide more visualization results on how this method outperforms others?
3 comments
Thank you for your clear answers. The statistical and visualization results provide more details about the method.
Thank you for your positive reassessment. We sincerely appreciate your helpful feedback, which has greatly helped us improve the paper.
Weakness
We provide additional results on ActivityNet Captions and Charades-CD in Tables 1–3 of this response. On ActivityNet Captions, S2PG improves SnAG across all reported metrics, increasing mIoU from 46.71% to 49.03% (+2.32 percentage points). This further supports the effectiveness of sequence-consistent candidate selection.
Table 1. ActivityNet Captions grounding results (%).
Following SVPTR, we additionally evaluate S2PG on Charades-CD. S2PG improves SnAG across all reported metrics on both the IID and OOD splits, increasing mIoU from 46.15% to 48.00% and from 36.50% to 38.66%, respectively. These gains support the effectiveness of sequence-consistent candidate selection under this setting.
Table 2. Charades-CD IID(%).
Table 3. Charades-CD OOD grounding results(%).
PRVG† results on Charades-CD is reproduced with the released code. S2PG also achieves the highest [email protected] and [email protected] scores among the listed methods on both Charades-CD splits.
Question
Our perturbation experiments can inform future VPG research by examining how existing methods exploit different types of paragraph information. These findings can guide future studies in identifying underutilized temporal-order cues and exploring how to use them to improve localization performance.
We then identify paragraph-level temporal order as an effective yet underexplored cue for connecting video sentence grounding (VSG) with paragraph-level joint grounding. Building on this finding, we propose S2PG, a lightweight, training-free framework that can be directly combined with existing VSG models. S2PG explores a new way to exploit temporal-order information by reformulating VPG as a sequence-level candidate-selection problem that balances sentence-level localization confidence with paragraph-level sequence consistency. This perspective can inspire future research toward more powerful VPG models.
Finally, S2PG (OSGNet) surpasses the previous VPG SOTA by 14.84% in mIoU on TACoS and 11.21% in [email protected] on Ego4D-GoalStep. S2PG can therefore serve as a strong VPG baseline for practical use and comparisons with future methods.
To address your request for additional visualizations showing how our method outperforms other methods, the figure available at this link presents a qualitative comparison of SnAG, S2PG (SnAG), and PRVG on a Charades-CD paragraph. SnAG predicts an earlier start for Q3 than for Q2, which is inconsistent with the temporal order of the paragraph. By penalizing start-time inversions during candidate reranking, S2PG selects a later candidate for Q3, restoring temporal consistency and yielding a closer match to the ground truth. Compared with PRVG, S2PG (SnAG) also produces predictions that align more closely with the ground truth, particularly for Q3 and Q4. Qualitative comparisons on TACoS and Ego4D-GoalStep are already included in Figure 5 and Figure 7 of the manuscript, respectively.