- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies training-free visual token compression for efficient VLLMs. It addresses the limitations of uniform token allocation and attention-based token selection, which overlook variations in scene complexity and temporal dynamics. It proposes DynVid, which combines adaptive budget allocation with dynamism-aware importance scoring and Facility Location-based token selection. Experiments across 8 benchmarks and 4 VLLMs show that DynVid preserves 99.3% relative accuracy at 10% average token retention on LLaVA-OneVision and achieves up to 9.49× prefilling speedup on Qwen2.5-VL.
Critical strengths
The paper addresses an important practical challenge in efficient video understanding. The proposed framework provides a systematic approach to reducing visual token redundancy through adaptive token budget allocation and representative token selection.
DynVid is training-free and applicable across different VLLM architectures, making it a flexible and practical solution with strong potential for integration into existing video understanding systems.
The experimental evaluation is extensive, covering 8 benchmarks, 4 VLLMs, and a wide range of compression ratios. The evaluation is further supported by component-wise ablation studies, efficiency analyses, and additional experiments with increased numbers of input frames.
Critical weaknesses
The core ideas behind DynVid have been explored in several closely related studies. Specifically, InfoMerge[1] already explores spectral entropy-based adaptive token allocation, MeToM[2] dynamically allocates token budgets based on video complexity, and FLoC[3] employs Facility Location for video token compression. These studies substantially overlap with the motivation and methodology of DynVid. This paper should clarify the key differences and provide comparisons with these methods.
The proposed dynamism score has an unexpected algebraic simplification. Eqs. (6) to (8) are equivalent to , meaning that the novelty and asymmetry terms reduce to a single minimum similarity measure. This raises questions about the necessity of the proposed two-component design. This paper should clarify its motivation and provide separate ablations for novelty and asymmetry.
The Facility Location formulation may suffer from negative importance weights. Since the attention and dynamism scores are normalized using z-score normalization, the resulting importance scores can be negative. Consequently, the objective may assign zero coverage to similar low-importance tokens while rewarding negative similarities. This paper should clarify how negative weights are handled and evaluate alternatives with nonnegative importance scores.
The soft fusion strategy may compromise temporal information. In Eqs. (13) and (14), tokens are aggregated based on feature similarity without explicit temporal constraints. This may mix information from different time steps and obscure temporal ordering, which is inconsistent with the paper's motivation of preserving temporal dynamics. This paper should clarify how temporal information is preserved during fusion
The computational scalability of DynVid is insufficiently evaluated. Appendix B shows that Facility Location selection requires time and memory per segment. Since temporal segmentation has no explicit maximum length, long continuous scenes may form a single large segment, leading to cubic worst-case time complexity under a fixed retention ratio. However, Table 2 only reports efficiency for 32-frame inputs, while the 160-frame experiments lack latency and memory measurements. This paper should provide compression latency, TTFT, peak GPU memory, and segment length statistics at larger frame counts, including stress tests on long continuous scenes.
The effectiveness of DynVid as a standalone compression method remains unclear. The main experiments adopt the inner-LLM query-guided pruning strategy from FlashVID. Although Figure 6 shows lower performance when this stage is removed, the number of tokens entering the LLM also changes. This paper should provide controlled comparisons with and without inner-LLM pruning under matched input token budgets.
[1] Liu, Xinxin, et al. "InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models." arXiv 2026.
[2] Wu, Zhuojie, et al. "MeToM: Metadata-Guided Token Merging for Efficient Video LLMs." CVPR 2026.
[3] Cho, Janghoon, et al. "Floc: Facility location-based efficient visual token compression for long video understanding." ICLR 2026.
Questions for authors
Plz refer to the weaknesses.
10 comments
Response to Reviewer LhhM (Part II)
Table 22. LLaVA-OneVision (average retention ratio of 10% for compressed methods)
| Variant | (Novelty weight) | Asymmetry weight | Video-MME | LongVideoBench | LVBench | TempCompass | Avg. score | Rel. Acc (%) |
|---|---|---|---|---|---|---|---|---|
| Asymmetry only | 0.0 | 1.0 | 56.8 | 56.7 | 37.1 | 64.2 | 53.70 | 98.3 |
| Weighted combination | 0.1 | 0.9 | 57.2 | 56.4 | 37.3 | 64.2 | 53.78 | 98.4 |
| Weighted combination | 0.25 | 0.75 | 57.7 | 56.9 | 38.0 | 64.5 | 54.28 | 99.3 |
| DynVid (equal weights) | 0.5 | 0.5 | 57.7 | 56.5 | 38.2 | 64.3 | 54.18 | 99.1 |
| Weighted combination | 0.75 | 0.25 | 57.5 | 56.1 | 38.3 | 63.9 | 53.95 | 98.7 |
| Weighted combination | 0.9 | 0.1 | 57.2 | 56.0 | 37.6 | 63.6 | 53.60 | 98.1 |
| Novelty only | 1.0 | 0.0 | 57.1 | 56.1 | 37.1 | 63.7 | 53.50 | 97.9 |
Table 23. Qwen2.5-VL (average retention ratio of 10% for compressed methods)
| Variant | (Novelty weight) | Asymmetry weight | Video-MME | LongVideoBench | LVBench | TempCompass | Avg. score | Rel. Acc (%) |
|---|---|---|---|---|---|---|---|---|
| Asymmetry only | 0.0 | 1.0 | 58.2 | 57.6 | 37.3 | 70.6 | 55.93 | 96.8 |
| Weighted combination | 0.1 | 0.9 | 58.3 | 57.5 | 37.6 | 71.0 | 56.10 | 97.1 |
| Weighted combination | 0.25 | 0.75 | 58.2 | 57.7 | 37.8 | 70.9 | 56.15 | 97.2 |
| DynVid (equal weights) | 0.5 | 0.5 | 58.4 | 58.2 | 37.8 | 71.1 | 56.38 | 97.6 |
| Weighted combination | 0.75 | 0.25 | 58.2 | 57.5 | 37.1 | 70.9 | 55.93 | 96.8 |
| Weighted combination | 0.9 | 0.1 | 58.2 | 57.6 | 37.5 | 70.6 | 55.98 | 96.9 |
| Novelty only | 1.0 | 0.0 | 58.2 | 57.5 | 37.2 | 70.6 | 55.88 | 96.8 |
W3. Negative importance weights.
Handling negative importance scores. We apologize for omitting the sigmoid step from the normalization description in the manuscript. Our implementation applies a sigmoid after z-score normalization, rather than using z-score alone. Specifically, in our anonymous code repository, line 52 of dynvid/compression_unified.py contains out = torch.sigmoid((x - mu) / sigma). The actual normalization is , which maps the scores to values between 0 and 1. Both attention and dynamism are normalized in this manner before combination, so the resulting importance weights are nonnegative. Consequently, multiplying an importance weight by a negative cosine similarity cannot produce a positive contribution; negative weighted similarities contribute zero through the clipping in the FL objective.
Normalization ablation. To evaluate the effect of normalization on importance-weighted selection, we compare z-score alone, which can produce negative importance weights, min–max normalization, and our implemented z-score plus sigmoid on four benchmarks using both backbones. As shown in Tables 24 and 25 below, z-score plus sigmoid improves the average score over z-score alone by 0.90 points on LLaVA-OneVision and 0.63 points on Qwen2.5-VL. With z-score alone, negative importance weights can reverse the intended similarity-based reward: negative cosine similarities may yield positive contributions, whereas positive similarities may yield negative contributions. This provides a possible explanation for its lower performance. Min–max normalization also outperforms z-score alone, matching the average score of our default normalization on LLaVA-OneVision (54.18) and achieving a slightly lower average score on Qwen2.5-VL (56.30 vs. 56.38). These results support the use of nonnegative normalized importance weights and suggest low sensitivity to the choice between min–max normalization and z-score plus sigmoid in the evaluated settings.
Table 24. Normalization ablation on LLaVA-OneVision (average retention ratio of 10%)
| Normalization | Video-MME | LongVideoBench | LVBench | TempCompass | Avg. score | Rel. Acc (%) |
|---|---|---|---|---|---|---|
| Z-score only | 57.1 | 55.9 | 36.2 | 63.9 | 53.28 | 97.5 |
| Min–max | 58.1 | 56.4 | 37.8 | 64.4 | 54.18 | 99.1 |
| Z-score + sigmoid (DynVid) | 57.7 | 56.5 | 38.2 | 64.3 | 54.18 | 99.1 |
Table 25. Normalization ablation on Qwen2.5-VL (average retention ratio of 10%)
| Normalization | Video-MME | LongVideoBench | LVBench | TempCompass | Avg. score | Rel. Acc (%) |
|---|---|---|---|---|---|---|
| Z-score only | 58.1 | 57.3 | 37.2 | 70.4 | 55.75 | 96.5 |
| Min–max | 58.5 | 57.9 | 38.0 | 70.8 | 56.30 | 97.5 |
| Z-score + sigmoid (DynVid) | 58.4 | 58.2 | 37.8 | 71.1 | 56.38 | 97.6 |
Response to Reviewer LhhM (Part III)
W4. Temporal information
Temporal information during soft fusion. Soft fusion updates the anchor features without changing their original positional information or temporal order. Each anchor remains associated with its original frame and position, while absorbing complementary information from non-anchor tokens through the similarity-weighted aggregation in Eqs. (13) and (14). Moreover, fusion is performed independently within each temporally coherent segment defined in Section 4.1, rather than across the entire video, preventing aggregation across segment boundaries. Thus, fusion retains the temporal structure of the selected anchors while incorporating additional within-segment context without increasing the token count.
W5. Scalability
Observed segment lengths. Table 26 below reports segmentation statistics over 900 unique Video-MME videos. As the sampled frame count increases from 64 to 160, the average number of segments per video increases from 29.75 to 70.72, while the mean segment length remains approximately 2.15–2.26 frames. The median segment length is 1 frame, and the 90th-percentile (P90) segment length is 4–5 frames, indicating that most segments remain short while some group multiple consecutive frames. Nevertheless, the maximum segment lengths are 54, 69, and 87 frames, respectively, showing that long segments can still occur.
Table 26. Temporal segment statistics on Video-MME with LLaVA-OneVision at different frame counts.
| Sampled frames | Unique videos | Total segments | Avg. segments per video | Mean segment length (frames) | Median segment length (frames) | P90 segment length (frames) | Maximum segment length (frames) |
|---|---|---|---|---|---|---|---|
| 64 | 900 | 26,774 | 29.75 | 2.15 | 1 | 4 | 54 |
| 128 | 900 | 51,528 | 57.25 | 2.24 | 1 | 5 | 69 |
| 160 | 900 | 63,648 | 70.72 | 2.26 | 1 | 5 | 87 |
Efficiency at larger frame counts. Table 27 below reports efficiency at 64, 128, and 160 frames. Under normal temporal segmentation, DynVid's compression latency is 65.7, 108.0, and 129.4 ms, respectively, lower than FlashVID's at all three frame counts. Its TTFT is 811.1, 1579.5, and 1963.4 ms, and the reported overall peak GPU memory remains close to FlashVID's. Together with the segment statistics, these results demonstrate practical efficiency when most segments remain short, without implying a hard bound on segment length or worst-case computational cost.
Table 27. Efficiency on LLaVA-OneVision at larger frame counts (average retention ratio of 10% for compressed methods).
| Frames | Method | Vision encoding time (ms) | Compression time (ms) | LLM Forward time (ms) | Prefilling time (ms) | TTFT (ms) | Peak GPU memory (MB) | TFLOPs |
|---|---|---|---|---|---|---|---|---|
| 64 | Vanilla | 584.8 | 0.0 | 1659.6 | 1659.6 | 2244.4 | 22735.4 | 113.4 |
| 64 | FlashVID | 584.8 | 76.7 | 163.9 | 240.6 | 825.4 | 22737.8 | 8.73 |
| 64 | DynVid with single-segment FL | 584.8 | 2113.2 | 160.3 | 2273.5 | 2858.3 | 22738.4 | 8.73 |
| 64 | DynVid (single segment, seed-and-block selection) | 584.8 | 104.2 | 160.3 | 264.5 | 849.3 | 22738.4 | 8.73 |
| 64 | DynVid | 584.8 | 65.7 | 160.6 | 226.3 | 811.1 | 22738.2 | 8.73 |
| 128 | Vanilla | 1167.4 | 0.0 | 4003.3 | 4003.3 | 5170.7 | 30126.8 | 290.0 |
| 128 | FlashVID | 1167.4 | 135.3 | 308.4 | 443.7 | 1611.1 | 30131.3 | 18.2 |
| 128 | DynVid with single-segment FL | 1167.4 | 15948.0 | 304.4 | 16252.4 | 17419.8 | 30131.7 | 18.2 |
| 128 | DynVid (single segment, seed-and-block selection) | 1167.4 | 456.3 | 304.4 | 760.7 | 1928.1 | 30131.7 | 18.2 |
| 128 | DynVid | 1167.4 | 108.0 | 304.1 | 412.1 | 1579.5 | 30131.9 | 18.2 |
| 160 | Vanilla | 1454.6 | 0.0 | 5432.5 | 5432.5 | 6887.1 | 33819.5 | 402.0 |
| 160 | FlashVID | 1454.6 | 162.1 | 382.7 | 544.8 | 1999.4 | 33825.9 | 23.2 |
| 160 | DynVid with single-segment FL | 1454.6 | 30798.7 | 380.6 | 31179.3 | 32633.9 | 33826.4 | 23.2 |
| 160 | DynVid (single segment, seed-and-block selection) | 1454.6 | 750.5 | 380.6 | 1131.1 | 2585.7 | 33826.4 | 23.2 |
| 160 | DynVid | 1454.6 | 129.4 | 379.4 | 508.8 | 1963.4 | 33826.4 | 23.2 |
Forced single-segment stress test. To examine the large-segment bottleneck directly, we also force all sampled frames into one segment and directly use FL-based seletion. Compression latency increases to 2113.2, 15,948.0, and 30,798.7 ms, confirming the substantial cost of full greedy FL selection on large segments. This occurs because each greedy step evaluates marginal coverage gains over the large segment-wide token set, and the number of selection steps also grows with the token budget, yielding cubic worst-case selection time at a fixed retention ratio.
We additionally evaluate an exploratory variant using a small number of exact greedy seeds followed by blockwise selection (seed-and-block selection), reducing compression latency to 104.2, 456.3, and 750.5 ms, respectively. These results show that full greedy FL is costly in the single-segment extreme, but computationally optimized selection strategies can substantially mitigate this bottleneck.
Response to Reviewer LhhM (Part I)
Thank you for your constructive comments. We have addressed all your comments below with additional experiments.
W1. More comparison with recent related studies
Comparison with closely related works. InfoMerge combines segment uniqueness and spectral-entropy-based representational richness for adaptive budget calculation, together with temporal redundancy estimation, attention-based token selection, and density-based merging. MeToM uses codec residuals and GoP packet sizes to guide spatial merging and per-frame budget allocation, followed by attention-guided merging inside the LLM. FLoC employs similarity-based Facility Location selection within temporal blocks using lazy greedy optimization.
Our contribution lies in the specific allocate-then-select design that couples complexity-adaptive allocation with dynamism-aware coverage selection, aiming to make every retained token count. ERA uses PR as an efficient surrogate for effective rank to determine each segment's budget from its representational complexity, without requiring codec metadata. DATS combines temporal novelty and forward/backward asymmetry with attention to estimate token importance, then performs FL-based anchor selection under the allocated budget to optimize importance-weighted content coverage. We use FL as an optimization tool, rather than claiming it as a novel algorithm. Together, ERA and DATS address two crucial questions in token compression: how many tokens to allocate to each segment and which tokens to retain within it.
Additional experimental comparisons. We could not locate publicly available official implementations of InfoMerge, MeToM, and FLoC, and therefore do not report direct experimental comparisons with these methods. To broaden our empirical evaluation, we reproduced OTPrune[1] and AOT[2] and compared them with DynVid on both backbones at retention ratios of 10% and 5%, as reported in Tables 8 and 9 of our response to Reviewer kxtr. DynVid achieves the highest average score in all four settings, providing additional evidence of its effectiveness across backbones and compression levels.
[1] OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport, CVPR2026
[2] Token reduction via local and global context optimization for efficient video large language models, CVPR2026
W2. The motivation and more ablations for novelty and asymmetry.
Algebraic simplification and motivation. For tokens with both adjacent frames available, Eqs. (6)–(8) indeed simplify to dyn(vj)=1−min(s−,s+). We use the two-component decomposition to distinguish different temporal-change patterns, rather than claiming that their sum is algebraically distinct from this expression. For example, Novelty can measure the absence of a strong match in either adjacent frame, whereas asymmetry can measure differences between forward and backward temporal correspondence. When both similarities are low, novelty is high but asymmetry can be low; when one similarity is high and the other is low, asymmetry is high while novelty can be low. The decomposition thus provides an interpretable motivation and enables separate evaluation of the two signals.
Ablation settings. To separate their contributions, we evaluate the weighted dynamism score dynα(vj)=α⋅novelty(vj)+(1−α)⋅asym(vj), varying α from 0 to 1 while keeping the remaining settings unchanged. We evaluate four benchmarks on LLaVA-OneVision and Qwen2.5-VL using 32 frames and an average retention ratio of 10%. Here, α=0 uses asymmetry only, α=1 uses novelty only, and α=0.5 gives the equal-weight combination.
Results. As shown in Tables 22 and 23 below, the default equal-weight combination achieves higher average scores than either component alone on both backbones. On LLaVA-OneVision, it achieves 54.18, compared with 53.70 for asymmetry alone and 53.50 for novelty alone; the highest average score is 54.28 at α=0.25. On Qwen2.5-VL, the equal-weight combination achieves the highest average score of 56.38, compared with 55.93 for asymmetry alone and 55.88 for novelty alone. These results support the complementary value of the two signals rather than a single universally dominant component. We use equal weights by default to avoid additional tuning.