- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TIDE, a training-free visual token compression framework for video temporal grounding with large vision-language models. It aims to preserve temporal evidence needed for event localization, rather than relying only on visual similarity or saliency. TIDE tracks cross-frame token correspondences to guide query-conditioned budget allocation, diverse anchor selection, and context fusion that discourages mixing across temporal states. Experiments on ActivityNet-Captions and Charades-STA report higher mIoU than the compared compression methods. The paper further evaluates the design through component ablations and additional VideoQA experiments.
Critical strengths
- Extensive experimental evaluation.
The authors evaluate TIDE across multiple backbones and token retention ratios on Charades-STA and ActivityNet-Captions, with additional VideoQA experiments. They further demonstrate the ablation studies and many analyses.
- An interesting focus on state transitions.
The idea of preserving state transitions during token compression for VTG is interesting. The paper emphasizes that precise localization relies on subtle state transitions rather than visual appearance alone.
Critical weaknesses
Insufficient analysis and justification of TIDE’s modules.
- Tables 3 and 4 show that the modules improve performance, but provide limited insight into how well each module serves its intended purpose. For example, TIDE still outperforms the compared compression methods without message passing in the reported setting. With several modules combined and limited analysis, it is difficult to identify which design choices are key to solving the stated problem and driving the gains.
- The paper also needs to better explain why the specific rules in these modules are necessary to address the stated problem, beyond describing how they use state-transition information. One way to improve this would be to first define key requirements for VTG compression and then explain which requirements each module addresses. This would help readers understand why each module is needed.
Questions for authors
-
Were the baseline latencies in Table 6 remeasured or taken from SemVID? Several values appear identical to those reported in SemVID, although Section 5.4 states that the timings were measured in the authors’ environment. Could the authors explain this and clarify whether the reported speedup is based on measurements obtained under the same conditions?
-
How does TIDE compare with FlashVID in terms of accuracy and efficiency? FlashVID achieves competitive grounding performance in Tables 1–2, but its latency is missing from Table 6. Could the authors report its compression and LLM prefilling times using the same backbone, input settings, and token retention ratio as TIDE?
-
How does TSF balance feature contextualization with preserving temporal contrast? In lines 052–053, the authors note that averaging across states can weaken temporal contrast, yet TSF aggregates adjacent-frame features before estimating transitions and states. While Table 4 shows a performance benefit from message passing, could the authors clarify when this aggregation helps correspondence estimation?
5 comments
Response 1 to W1: Connecting task requirements, module behavior, and localization gains
Context recovery, the shared temporal representation, and message passing provide substantial localization gains on ANet. New diagnostics connect allocation, selection, and fusion to their intended roles. Following your suggestion, we organize the design around four requirements:
| Requirement | Module and design rationale |
|---|---|
| Compare content across frames | TSF: spatial-semantic affinities guide aggregation and predecessor matching, providing a shared reference for change, trajectories, and states. |
| Balance visual complexity and temporal evidence within a fixed budget | EDA: effective rank gives complex static frames an allocation signal; transition terms emphasize change. A within-unit two-sided maximum shares the signal across adjacent observations, with integer quotas preserving the budget exactly. |
| Select informative anchors while controlling repetition | ESS: the predecessor-similarity discount addresses repetition across frames; MMR promotes feature diversity within each frame. |
| Recover discarded context while respecting state differences | BPF: local soft top-k routing recovers context. A query-and-change gate controls the state penalty, favoring compatible destinations while retaining fallback routes. |
EDA decides how many tokens each frame receives, ESS decides which tokens remain, and BPF decides how unselected context contributes to them. TSF supplies their common temporal reference.
Localization contributions. Tables 3–4 use Qwen3-VL-4B on ANet at 12.5% retention:
| Variant | mIoU | Full TIDE minus variant |
|---|---|---|
| Full TIDE | 42.13 | — |
| w/o Context Fusion | 40.19 | +1.94 |
| w/o TSF | 40.61 | +1.52 |
| w/o Message Passing | 40.71 | +1.42 |
| w/o Temporal Discount | 40.96 | +1.17 |
| w/o Trajectory | 40.97 | +1.16 |
| Rank-only allocation | 41.12 | +1.01 |
Removing TSF disables the shared representation and its dependent signals. Removing context fusion retains allocation and anchor selection. Removing message passing uses raw features and recomputes trajectories, states, and downstream computations. Each row measures full-pipeline performance under the specified intervention.
As you observed, the remaining pipeline performs well without message passing. Its additional contribution is measured within that pipeline: 40.71 to 42.13 mIoU. On Qwen3-VL-4B/Charades-STA, disabling TSF, using rank-only allocation, removing temporal discount, or removing context fusion lowers mIoU by 0.60, 0.71, 0.75, or 0.30 points.
Behavior diagnostics. We use Qwen3-VL-4B at 12.5% retention, with a fixed 5,000-query ANet subset and all 3,720 Charades-STA queries:
- EDA: relative to uniform allocation within the same unit, quotas before/after ANet GT starts are 1.12/1.09, and before/after ends 1.07/1.10. Charades-STA values are 1.01/1.03 and 1.05/0.99. Thus, ANet internal boundary neighborhoods receive 7%–12% more capacity on average.
- ESS: with allocation and MMR fixed, temporal discount changes adjacent-frame position reuse from 0.51 to 0.50 on ANet and 0.51 to 0.49 on Charades-STA. Mean selected-token change increases from 0.11 to 0.12 and 0.07 to 0.08; trajectory reuse remains approximately 0.85/0.75.
- BPF: with anchors fixed, the fraction of fusion weight crossing GT boundaries for positively gated sources within ±1 visual time step falls from 18.48% to 14.69% on ANet and 19.75% to 16.14% on Charades-STA. Admission rates change from 96.83% to 95.25% and 99.66% to 99.49%; zero-gate routing is unchanged.
Finally, w/o fusion / α=0 / full TIDE use the same selected anchors and yield 40.19/42.08/42.13 mIoU on ANet and 53.39/53.30/53.69 on Charades-STA. The α=0 comparison retains the other fusion rules while admission and routing weights respond to α. These controls separate context recovery from the state penalty: on ANet, recovery adds 1.89 points and the penalty another 0.05; on Charades-STA, the penalty adds 0.39 points. We will present performance and behavior results together to explain each design choice.
Response 2 to Q1 + Q2: Timing provenance, a unified retest, and comparison with FlashVID
Under the unified timing protocol, TIDE has a reported compression-plus-prefill total of 291.43 ms, giving 4.44× over Full-token from the stage statistics. Relative to FlashVID, it adds 13.08 ms (4.70%) while achieving 5.36 higher mIoU in the ANet/Qwen3-VL-4B main-table comparison.
Provenance and retest. The original Full-token, VisionZip, FastVID, and SemVID timing rows were quoted from SemVID Table 6 (L40); TIDE was measured by us on A800. The original 4.7× ratio combined these sources. We will correct the source description and replace the table with our retest of all six methods, including FlashVID, under the same hardware, software, and input settings.
The retest is conducted under the same configuration and experimental environment, using Qwen3-VL-4B, 12.5% nominal retention, three warm-up runs, and five repetitions. Compression is the method-specific net cost after subtracting shared Full-token visual/assembly overhead. Total sums the reported compression and prefill medians across videos; speedup divides Full-token Total by method Total. The scope covers compression and prefill, excluding video decoding, vision encoding, and autoregressive generation.
| Method | Compression (ms) | Prefill (ms) | Total (ms; speedup) | mIoU | (%) |
|---|---|---|---|---|---|
| Full-token | — | 1293.36 | 1293.36 (1.00×) | 42.93 | 100.00 |
| VisionZip | 556.73 | 201.14 | 757.87 (1.71×) | 32.37 | 75.40 |
| FastVID | 20.52 | 183.90 | 204.42 (6.33×) | 33.94 | 79.06 |
| FlashVID | 84.55 | 193.80 | 278.35 (4.65×) | 36.77 | 85.65 |
| SemVID | 65.16 | 191.82 | 256.98 (5.03×) | 38.49 | 95.40 |
| TIDE | 97.53 | 193.90 | 291.43 (4.44×) | 42.13 | 98.14 |
Timing comes from the new 200-video test. Accuracy and relative Full-token scores (%) come from the ANet main table. Italic SemVID accuracy entries are quoted from its paper and use its own Full-token baseline; the other accuracy entries are our evaluations.
FlashVID and TIDE take 84.55/97.53 ms for compression and 193.80/193.90 ms for prefill. Their totals differ by 13.08 ms. This quantifies the additional cost associated with TIDE's localization gains. Methods use the same nominal retention; actual visual-token counts depend on each implementation. FlashVID retains more visual tokens than TIDE in this timing batch.
Accuracy across settings. TIDE has higher mIoU than FlashVID in all 12 main-table settings: two datasets, three backbones, and two retention ratios. At 12.5% retention:
| Dataset | Backbone | FlashVID mIoU | TIDE mIoU | Gain |
|---|---|---|---|---|
| ANet | Qwen3-VL-4B | 36.77 | 42.13 | +5.36 |
| ANet | Qwen3-VL-8B | 35.47 | 42.75 | +7.28 |
| ANet | Qwen3.5-4B | 40.24 | 45.28 | +5.04 |
| Charades-STA | Qwen3-VL-4B | 52.82 | 53.69 | +0.87 |
| Charades-STA | Qwen3-VL-8B | 54.92 | 55.34 | +0.42 |
| Charades-STA | Qwen3.5-4B | 52.38 | 54.75 | +2.37 |
On the two Qwen3-VL backbones, TIDE at 12.5% retention also exceeds FlashVID at 25% on ANet (42.13/42.75 vs. 39.43/39.63). We will replace Table 6 with the unified retest, add FlashVID, and update the compression-plus-prefill speedup from 4.7× to 4.44× throughout the paper. Fixed-shape module profiling will retain its separate measurement conditions.
Summary: Module evidence, unified timing, and analysis of temporal contextualization
We sincerely thank the reviewer for the careful reading and constructive suggestions, and for recognizing our experimental coverage and focus on state transitions. The questions about component contributions, timing comparisons, and temporal smoothing helped us strengthen the experimental evidence and clarify the method. Our main responses are:
The modules address distinct compression decisions, with their contributions measured separately (W1). VTG compression must allocate limited capacity, choose informative observations, and recover the context needed to interpret them. EDA, ESS, and BPF handle these decisions using TSF's shared temporal reference. The ANet removal studies identify context recovery and temporal feature construction as major contributors; adding message passing to the remaining pipeline improves mIoU by 1.42 points. We pair these ablations with allocation, repetition, and GT-boundary routing diagnostics to explain how each rule serves its intended purpose. See Response 1.
The updated speedup comes from a same-condition retest (Q1). The original baseline timings were quoted from SemVID on L40, while TIDE was measured by us on A800. We have remeasured every method under one hardware, software, and input configuration. The replacement result is 4.44× for compression plus LLM prefill, with Total defined as the sum of the reported stage medians. We will update Table 6 and the corresponding statements throughout the paper. See Response 2.
TIDE provides higher grounding accuracy with additional compression cost (Q2). The new retest includes FlashVID's compression and prefill times. On Qwen3-VL-4B/ANet at 12.5% nominal retention, TIDE gains 5.36 mIoU in the main accuracy evaluation and adds 13.08 ms in the timing test. Prefill times are nearly identical, so the additional cost is concentrated in compression. The accuracy advantage also holds across the remaining main-table settings. See Response 2.
Affinity-guided contextualization retains useful temporal evidence beyond plain smoothing (Q3). TSF retains half of each token's own feature and aggregates similar content from adjacent frames before estimating changes and states. To verify that this contextualization retains localization cues, we compare it with fixed-position temporal averaging using the same self weight and temporal neighborhood. TIDE improves mIoU over that control by 0.38/1.53 points on ANet/Charades-STA. Holding contextualized features fixed also shows higher GT-boundary recall with matched predecessors than with fixed positions. Together, these results support preserving grounding evidence through selective contextualization and correspondence-based change estimation. See Response 3.
The added experiments, method explanations, and corresponding revisions will be marked in blue in the updated PDF. We hope this additional evidence addresses your concerns and would be glad to discuss any remaining questions.