- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TopoAtom, a training-free video temporal grounding method that constructs variable-length semantic atoms based on video structure and query relevance. It builds a visual similarity hierarchy and applies query-conditioned pruning to select atoms. Consecutive atoms are combined into candidate segments, which are ranked using query relevance and candidate persistence. The method achieves improvements across multiple benchmarks and frozen backbones, supported by ablations and theoretical analysis.
Critical strengths
-
The paper provides thorough experimental and theoretical analyses. It includes component ablations, candidate oracle performance, hyperparameter sensitivity, runtime measurements, and qualitative examples. The appendix further analyzes the mathematical properties of the proposed mechanisms, providing useful insights into the method.
-
The method demonstrates consistent improvements across multiple benchmarks and frozen backbones. Experiments on Charades-STA, ActivityNet-Captions, QVHighlights, and the additional long-video evaluation on TACoS support its effectiveness across different datasets and settings.
Critical weaknesses
-
The computational overhead is not sufficiently evaluated. TopoAtom encodes multiple clips and grouped segments with Qwen, potentially introducing substantial computational cost. The claim that node similarities can be computed in a few batched forward passes (Section 3.3) does not necessarily imply low computation or memory usage. Appendix A.4 reports runtime but lacks detailed encoding settings, batching configurations, and GPU memory measurements, making practical efficiency and long-video scalability difficult to assess.
-
The core selection mechanisms lack sufficient justification for their grounding benefits. It is unclear why comparing a node's query similarity with those of its descendants should select appropriate temporal granularity rather than excessively splitting segments. Joint encoding produces scores that differ from averaged clip similarities, but this alone does not explain why the pruning rule should favor semantically appropriate segments. Similarly, the paper does not sufficiently explain why structural coherence should improve query-relevant localization or which limitations of relevance-only ranking it addresses. These gaps limit understanding of why the proposed mechanisms work.
Questions for authors
-
How significant is the computational overhead of the proposed method? Since TopoAtom requires additional segment encoding with Qwen, could the authors clarify the number of encoded segments, batching configurations, and peak GPU memory usage?
-
Why should query-conditioned pruning select appropriate temporal granularity? Joint encoding produces similarity scores that differ from those of individual clips, but it remains unclear when a semantically complete segment would score higher than its descendants. What prevents the method from excessively splitting segments?
-
Why does the pruning mechanism work well for queries about both background context and foreground interactions, as illustrated in Section 4.4? The paper does not clearly explain which aspects of the pruning rule lead to better localization than TAG in these examples.
-
What limitations of relevance-only ranking does candidate persistence address? The paper introduces candidate persistence to complement query relevance, but does not clearly explain what errors arise from relevance-only ranking or why structural coherence should help correct them. While Table 4 reports changes in candidate rankings and Appendix A.3 presents persistence distributions, could the authors clarify when and why structural coherence helps or hurts localization accuracy?
-
Why are different baselines used in Tables 1 and 2? Could the authors clarify the baseline selection criteria, including which results are reproduced and why some relevant baselines are absent from QVHighlights?
4 comments
Dear Reviewer eXFW,
Thank you again for the time and effort. As the rebuttal period draws to a close, we wanted to briefly follow up. If you have any remaining questions about our response or the revised manuscript, we’d be happy to clarify or discuss them.
If the authors provide a latency comparison with other VTG methods under comparable conditions, I would feel more confident in maintaining my weak accept rating. However, I suspect that some existing methods may have substantially lower latency, potentially below one second, compared with the reported latency of over 10 seconds for TopoAtom. I find the proposed idea interesting, but its computational cost raises concerns about practical applicability. Further efficiency improvements may be needed, and I encourage the authors to explicitly discuss this limitation.
Follow-up on the inference latency
Thank you for your following up. We chose TAG as a strong open-source baseline because both methods organize video content into temporal units for grounding. TAG uses query-independent clustering with a fixed cluster count, while TopoAtom uses the query to adapt the granularity of video parsing. This comparison helps quantify the inference cost associated with our design.
Charades-STA. Figure 7 reports total latency and a breakdown by stage. TopoAtom with Qwen3-VL-Embedding 8B takes 5.1 s, compared with 4.0 s for TAG. Most of the difference comes from feature encoding.
Long-video comparison. The latencies above 10 s were measured on ActivityNet-Captions videos longer than two minutes and on TACoS. We added TAG results on the same videos in Table 10.
| Dataset | Method | Batch size | Latency (s) | Peak GPU memory (GB) | Avg. atoms or clusters | Avg. encoded inputs |
|---|---|---|---|---|---|---|
| ActivityNet-Captions (>2 min) | TopoAtom (8B) | 4 | 10.3 | 22.6 | 29.4 | 79.2 |
| ActivityNet-Captions (>2 min) | TAG | 256 | 6.6 | 14.5 | 9 | 1188.1 |
| TACoS | TopoAtom (8B) | 2 | 16.9 | 22.7 | 35.9 | 135.5 |
| TACoS | TAG | 256 | 10.9 | 14.8 | 9 | 2032.6 |
For TopoAtom, batch size and encoded inputs refer to hierarchy-induced video segments. For TAG, they refer to individual frames sampled at 3 fps. TAG uses nine clusters in both evaluations.
Limitations and future directions. The additional time is an overall reasonable trade-off for the query-adaptive parsing. Note that for a video with base clips, TopoAtom encodes at most distinct video segments to score the hierarchy, including the base clips (added in Section 3.3). We also added a discussion at the end of Section 4.4 stating that the latency may limit applicability in some scenarios that extremely sensitive to the latency. We also outline a direction for improving efficiency by screening hierarchy nodes with inexpensive cues before joint encoding.
Follow-up on Our Rebuttal
Dear Reviewer eXFW,
Thank you again for your questions about the design and evaluation.
We expanded the explanation of when query-conditioned pruning preserves a broader segment or splits it further (Sections 3.3 and 4.4). We also added examples of when candidate persistence helps or hurts (Fig. 5) and reported batch sizes, encoded-segment counts, latency, and peak GPU memory for long videos (Table 10). Our reply also explains how we selected baselines for each benchmark.
If any concern remains, we would be glad to address it in the remaining discussion period.