- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DART, a training-free framework for video token compression based on budgeted spatiotemporal reconstruction. DART progressively partitions the video feature volume using temporal (T), spatial (XY), and joint spatiotemporal (TXY) refinements, selecting each operation according to its attention-weighted reconstruction-error reduction per additional token. Adaptive-T further optimizes temporal split boundaries. Experiments on LLaVA-OneVision-7B and 0.5B demonstrate competitive accuracy under 10%–25% visual-token retention.
Critical strengths
S1. Clear motivation and principled formulation. The paper addresses the different spatial and temporal resolution requirements of video content and formulates token allocation through a unified, budget-aware reconstruction objective.
S2. Training-free and flexible compression. DART operates before LLM prefilling without modifying pretrained parameters. Its query-independent design also allows compressed video representations to be reused across multiple questions.
S3. Strong accuracy under aggressive compression. DART achieves 59.1% average accuracy at 25% token retention on LLaVA-OneVision-7B, compared with 58.4% for the full-token baseline, and maintains competitive performance at 10% retention.
Critical weaknesses
W1. Limited algorithmic novelty over existing structured compression methods. The core framework combines established ideas in adaptive tree partitioning, weighted reconstruction, and greedy budget allocation. Although the paper distinguishes DART from similarity-based merging, its algorithmic advances over prior structured spatiotemporal methods, particularly ForestPrune, remain insufficiently established. Moreover, the independent contributions of Adaptive-T and TXY refinement are relatively modest in Table 5.
W2. Missing end-to-end efficiency evaluation. The reported 77.1%–90.3% reductions are analytical estimates of LLM-prefilling FLOPs, excluding attention extraction, iterative partition planning, reconstruction-error evaluation, and token pooling. Without actual compression latency and end-to-end measurements, the practical efficiency benefit remains unclear.
W3. Small performance margins and limited baseline comparability. At 25% retention, DART improves average accuracy over HoliTom and FlashVID by only 0.3 and 0.6 percentage points, respectively. Furthermore, Appendix B.3 confirms that baseline results are taken from prior publications rather than reproduced under a unified evaluation pipeline. These small margins may therefore be sensitive to implementation and evaluation differences.
W4. Limited cross-architecture validation. Experiments are restricted to two LLaVA-OneVision model scales. Since DART relies on SigLIP attention and a specific spatial feature layout, its effectiveness on other Video-LLM architectures remains unverified.
Questions for authors
Q1. Algorithmic novelty: Beyond replacing similarity-based merging with reconstruction-guided partitioning, what specifically distinguishes DART from ForestPrune and other adaptive spatiotemporal tree methods? Could the authors clarify the independent algorithmic contributions of axis-adaptive refinement and Adaptive-T?
Q2. Practical efficiency: Could the authors report compression latency broken down by attention extraction, partition planning, and pooling, together with actual TTFT and end-to-end speedups? How does DART compare with HoliTom and FlashVID under identical hardware and inference settings?
Q3. Baseline reliability: Given the small accuracy margins, could the authors evaluate the strongest baselines under a unified pipeline and report the variability or statistical significance of the improvements?
Q4. Generalization: How would DART adapt to architectures such as Qwen2.5-VL or Qwen3-VL, which use different visual tokenization and feature-processing pipelines?
6 comments
Official Comment by Authors
W2&Q2. Practical Efficiency and Compression Overhead.
We measure latency under identical settings: LLaVA-OneVision-7B, one RTX 3090, 32 frames, FP16/SDPA, batch size 1, and greedy generation of exactly 32 tokens. We evaluate 100 fixed MVBench questions over five rounds, with 10 warm-up requests per method per round. Each request independently performs video decoding, visual encoding, compression, and generation. We report the median across five repeats per question, followed by the median across questions. All latencies are in milliseconds.
| Method | Before LLM Retained Ratio | Visual Prep Latency ↓ | Planning Latency ↓ | Prefilling Latency ↓ | TTFT ↓ | End-to-End Latency ↓ | End-to-End Speedup ↑ | Avg. Score ↑ |
|---|---|---|---|---|---|---|---|---|
| Full | 100% | 618.7 | – | 1615.6 | 3055.1 | 3787.1 | 1.00× | 58.4 |
| HoliTom | 25% | 681.0 | – | 390.8 | 1894.5 | 2581.2 | 1.47× | 58.8 |
| EarlyTom | 25% | 496.7 | – | 391.1 | 1720.9 | 2407.9 | 1.57× | 58.2 |
| FlashVID | 25% | 659.9 | – | 333.0 | 1810.5 | 2496.9 | 1.52× | 58.5 |
| DART (Ours) | 25% | 740.1 | 118.9 | 422.3 | 1972.3 | 2657.1 | 1.43× | 59.1 |
DART’s compression costs are broken down below and included in the reported visual-preparation and end-to-end latencies.
| Component | Latency |
|---|---|
| Attention-based importance extraction, incremental overhead¹ | 3.94 ms |
| Partition planning: feature transfer, normalization, prefix statistics, candidate search, and reconstruction-error evaluation | 118.9 ms |
| Pooling: native spatial downsampling and region pooling | 3.41 ms |
¹ Attention-extraction overhead is measured using three paired extraction-on/off vision forwards per video on the same 100 videos, isolating the additional cost of obtaining importance maps from the encoder’s existing attention computation.
“–” denotes no separately isolated planning stage. TTFT spans video loading through first-token selection; end-to-end latency additionally includes the remaining generation and request finalization. Independently aggregated medians need not sum exactly.
At 25% retention, DART reduces prefilling latency, TTFT, and end-to-end latency by 73.9%, 35.4%, and 29.8% relative to Full, respectively, achieving a 1.43× end-to-end speedup with compression overhead included. It also achieves the highest four-benchmark average accuracy among the compared methods (59.1%, versus 58.4% for Full), demonstrating practical efficiency gains while preserving accuracy.
W3&Q3. Baseline Reliability and Evaluation Comparability
We additionally evaluate HoliTom, EarlyTom, and FlashVID under our unified LLaVA-OneVision-7B evaluation pipeline across all four benchmarks and retention ratios. The complete results are reported below. “Avg. Score” denotes the unweighted average across the four benchmarks, and “Relative” is normalized by the Full baseline’s average.
| Method | Pre-LLM retention | MVBench ↑ | EgoSchema ↑ | LongVideoBench ↑ | Video-MME ↑ | Avg. Score ↑ | Relative (%) ↑ |
|---|---|---|---|---|---|---|---|
| Full | 100% | 58.3 | 60.4 | 56.4 | 58.6 | 58.4 | 100 |
| HoliTom | 25% | 58.0 | 61.0 | 56.4 | 58.9 | 58.6 | 100.3 |
| EarlyTom | 25% | 57.8 | 60.3 | 56.2 | 58.8 | 58.3 | 99.8 |
| FlashVID | 25% | 58.0 | 60.2 | 57.6 | 58.3 | 58.5 | 100.2 |
| DART | 25% | 58.4 | 61.3 | 57.3 | 59.3 | 59.1 | 101.2 |
| HoliTom | 20% | 58.4 | 60.9 | 56.5 | 58.7 | 58.6 | 100.3 |
| EarlyTom | 20% | 57.6 | 60.4 | 55.1 | 58.1 | 57.8 | 99.0 |
| FlashVID | 20% | 57.8 | 60.1 | 57.3 | 58.3 | 58.4 | 100 |
| DART | 20% | 57.9 | 61.4 | 57.3 | 59.3 | 59.0 | 101 |
| HoliTom | 15% | 58.1 | 60.8 | 55.4 | 57.7 | 58.0 | 99.3 |
| EarlyTom | 15% | 57.1 | 60.4 | 54.5 | 57.5 | 57.4 | 98.3 |
| FlashVID | 15% | 57.3 | 60.0 | 57.1 | 57.4 | 58.0 | 99.3 |
| DART | 15% | 58.0 | 61.2 | 56.1 | 57.8 | 58.3 | 99.8 |
| HoliTom | 10% | 57.7 | 61.2 | 55.6 | 56.9 | 57.9 | 99.1 |
| EarlyTom | 10% | 55.8 | 59.4 | 53.1 | 55.0 | 55.8 | 95.5 |
| FlashVID | 10% | 56.9 | 59.7 | 54.2 | 56.4 | 56.8 | 97.3 |
| DART | 10% | 57.9 | 60.8 | 56.0 | 57.4 | 58.0 | 99.3 |
Under this unified evaluation, DART achieves the highest average accuracy at all four retention ratios. At 25% retention, it exceeds HoliTom, EarlyTom, and FlashVID by 0.5, 0.8, and 0.6 percentage points, respectively. These results show that DART’s average-performance advantage persists after reproducing the baselines within our pipeline, addressing the concern about cross-publication comparability.
Official Comment by Authors
W4&Q4. Generalization to Different Visual Architectures.
We implement DART on Qwen3-VL-2B-Instruct, retaining the reconstruction objective, budget-normalized action selection, and Adaptive-T. The architecture-specific adaptation consists of:
- Grid and importance: use Qwen’s post-merger temporal–spatial feature grid and derive importance from Qwen vision self-attention, aggregated to that grid.
- Feature integration: apply the same partition and pooling weights to the final visual features and aligned DeepStack features.
- Sequence integration: update video-token placeholders, timestamp grouping, and positional indices to accommodate the compressed sequence.
The complete 32-frame results are shown below. “Avg. Score” is the unweighted four-benchmark average; “Relative” is computed from the displayed averages relative to Full.
| Method | Pre-LLM retention | MVBench | EgoSchema | LongVideoBench | Video-MME | Avg. Score | Relative (%) |
|---|---|---|---|---|---|---|---|
| Full | 100% | 61.8 | 53.9 | 53.3 | 57.4 | 56.6 | 100 |
| HoliTom | 25% | 60.5 | 52.8 | 51.0 | 55.8 | 55.0 | 97.2 |
| EarlyTom | 25% | 60.2 | 52.7 | 50.4 | 54.5 | 54.5 | 96.3 |
| FlashVID | 25% | 59.1 | 50.9 | 52.1 | 55.5 | 54.4 | 96.1 |
| DART | 25% | 60.0 | 54.1 | 52.4 | 55.9 | 55.6 | 98.2 |
| HoliTom | 20% | 60.4 | 52.5 | 50.2 | 54.7 | 54.5 | 96.3 |
| EarlyTom | 20% | 59.0 | 51.8 | 50.1 | 54.1 | 53.8 | 95.1 |
| FlashVID | 20% | 58.7 | 50.7 | 51.9 | 54.4 | 53.9 | 95.2 |
| DART | 20% | 59.8 | 53.5 | 52.4 | 54.7 | 55.1 | 97.3 |
| HoliTom | 15% | 58.9 | 51.8 | 49.4 | 54.1 | 53.6 | 94.7 |
| EarlyTom | 15% | 56.9 | 50.6 | 49.4 | 53.1 | 52.5 | 92.8 |
| FlashVID | 15% | 57.9 | 49.8 | 51.3 | 53.5 | 53.1 | 93.8 |
| DART | 15% | 58.8 | 52.4 | 52.0 | 53.6 | 54.2 | 95.8 |
| HoliTom | 10% | 57.2 | 49.9 | 48.9 | 52.4 | 52.1 | 92.0 |
| EarlyTom | 10% | 56.2 | 48.7 | 48.8 | 51.0 | 51.2 | 90.5 |
| FlashVID | 10% | 56.4 | 49.0 | 50.4 | 52.1 | 52.0 | 91.9 |
| DART | 10% | 57.3 | 50.9 | 51.0 | 52.7 | 53.0 | 93.6 |
DART achieves the highest average accuracy at all four retention ratios, exceeding the strongest baseline by 0.6, 0.6, 0.6, and 0.9 percentage points, respectively. These results support transfer of its reconstruction-guided allocation mechanism beyond SigLIP and LLaVA-OneVision.
Official Comment by Authors
We thank the reviewer for the thoughtful assessment and constructive feedback, and for recognizing our clear motivation, training-free design, and strong performance under aggressive compression. Below, we clarify the algorithmic contributions and distinctions from ForestPrune, and present additional experiments on practical efficiency, baseline comparability, and cross-architecture generalization.
W1&Q1. Algorithmic Novelty and Distinction from ForestPrune.
At a high level, DART follows a fundamentally different compression paradigm from ForestPrune and related token-compression methods. These methods typically start from fine-grained visual tokens and reduce their number through token merging, grouping, or pruning. ForestPrune, for example, organizes frame-level tokens into cross-frame token forests and then compresses them through node pruning.
DART instead performs top-down spatiotemporal splitting. It starts from a single cuboid covering the complete video feature volume and recursively splits this cuboid into smaller regions along the temporal axis (T), the spatial axes (XY), or both jointly (TXY). The process continues until the token budget is reached, and each final leaf cuboid is represented by one pooled token. Consequently, the output is a complete and non-overlapping partition of the original spatiotemporal feature grid.
This difference changes the basic budget-allocation problem. Merging- or pruning-based methods decide which existing tokens should be combined or removed. DART decides where additional resolution should be created by splitting the current regions. At each step, candidate splits are ranked according to their importance-weighted reconstruction-error reduction per additional token. The token budget is therefore allocated to the regions and axes that provide the largest reconstruction benefit, while temporal and spatial resolution can be increased independently.
Our novelty is not tree construction or greedy optimization in isolation. It lies in coupling top-down spatiotemporal splitting with reconstruction-guided budget allocation and axis selection. This formulation retains the spatial and temporal support of each final region in the partition and explicitly controls how the token budget is distributed across the video volume. Adaptive-T is a boundary-selection mechanism within this top-down framework: after a temporal split is selected, it chooses the temporal boundary that gives the largest reconstruction benefit. Thus, DART provides a resolution-allocation mechanism based on recursively refining a complete spatiotemporal volume, which is conceptually distinct from reducing a set of fine-grained tokens through merging or pruning.