acceptodds
Under review as a conference paper at ICLR 2027

DART: Decoupled Axis-Adaptive Reconstruction Trees for Budgeted Video Token Compression

Abstract

Video large language models incur substantial computational costs when process- ing dense frame-wise visual tokens. Compressing these tokens under a fixed bud- get requires deciding not only which regions to preserve, but also whether to retain temporal variation, spatial detail, or both. We formulate this decision as budgeted, attention-weighted reconstruction of the video feature volume and propose Decou- pled Axis-Adaptive Reconstruction Trees (DART). DART represents the video as a partition of contiguous spatiotemporal regions, with one pooled token per re- gion. A greedy planner applies temporal, spatial, and spatiotemporal partition- ing to every candidate region, evaluates the reconstruction error of each resulting partition with respect to the original features within the region, and selects the re- gion and partitioning strategy that yield the greatest reconstruction-error reduction per additional token. Adaptive-T further selects content-dependent boundaries for temporal-only splits, while spatial and spatiotemporal splits use midpoint bound- aries. The resulting partition guides importance-weighted pooling of the original features before language-model prefilling, without query conditioning or updates to pretrained parameters. Averaged over four video-understanding benchmarks, DART retains 101.2% and 102.7% of the corresponding full-token baseline ac- curacy on LLaVA-OneVision-7B and 0.5B, respectively, while using only 25% of the visual tokens. These results support budgeted region refinement as an effective approach to allocating spatiotemporal resolution for video token compression.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
video large language models, visual token compression, spatiotemporal token compression, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A47MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DART, a training-free framework for video token compression based on budgeted spatiotemporal reconstruction. DART progressively partitions the video feature volume using temporal (T), spatial (XY), and joint spatiotemporal (TXY) refinements, selecting each operation according to its attention-weighted reconstruction-error reduction per additional token. Adaptive-T further optimizes temporal split boundaries. Experiments on LLaVA-OneVision-7B and 0.5B demonstrate competitive accuracy under 10%–25% visual-token retention.

Critical strengths

S1. Clear motivation and principled formulation. The paper addresses the different spatial and temporal resolution requirements of video content and formulates token allocation through a unified, budget-aware reconstruction objective.

S2. Training-free and flexible compression. DART operates before LLM prefilling without modifying pretrained parameters. Its query-independent design also allows compressed video representations to be reused across multiple questions.

S3. Strong accuracy under aggressive compression. DART achieves 59.1% average accuracy at 25% token retention on LLaVA-OneVision-7B, compared with 58.4% for the full-token baseline, and maintains competitive performance at 10% retention.

Critical weaknesses

W1. Limited algorithmic novelty over existing structured compression methods. The core framework combines established ideas in adaptive tree partitioning, weighted reconstruction, and greedy budget allocation. Although the paper distinguishes DART from similarity-based merging, its algorithmic advances over prior structured spatiotemporal methods, particularly ForestPrune, remain insufficiently established. Moreover, the independent contributions of Adaptive-T and TXY refinement are relatively modest in Table 5.

W2. Missing end-to-end efficiency evaluation. The reported 77.1%–90.3% reductions are analytical estimates of LLM-prefilling FLOPs, excluding attention extraction, iterative partition planning, reconstruction-error evaluation, and token pooling. Without actual compression latency and end-to-end measurements, the practical efficiency benefit remains unclear.

W3. Small performance margins and limited baseline comparability. At 25% retention, DART improves average accuracy over HoliTom and FlashVID by only 0.3 and 0.6 percentage points, respectively. Furthermore, Appendix B.3 confirms that baseline results are taken from prior publications rather than reproduced under a unified evaluation pipeline. These small margins may therefore be sensitive to implementation and evaluation differences.

W4. Limited cross-architecture validation. Experiments are restricted to two LLaVA-OneVision model scales. Since DART relies on SigLIP attention and a specific spatial feature layout, its effectiveness on other Video-LLM architectures remains unverified.

Questions for authors

Q1. Algorithmic novelty: Beyond replacing similarity-based merging with reconstruction-guided partitioning, what specifically distinguishes DART from ForestPrune and other adaptive spatiotemporal tree methods? Could the authors clarify the independent algorithmic contributions of axis-adaptive refinement and Adaptive-T?

Q2. Practical efficiency: Could the authors report compression latency broken down by attention extraction, partition planning, and pooling, together with actual TTFT and end-to-end speedups? How does DART compare with HoliTom and FlashVID under identical hardware and inference settings?

Q3. Baseline reliability: Given the small accuracy margins, could the authors evaluate the strongest baselines under a unified pipeline and report the variability or statistical significance of the improvements?

Q4. Generalization: How would DART adapt to architectures such as Qwen2.5-VL or Qwen3-VL, which use different visual tokenization and feature-processing pipelines?

6 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the thoughtful assessment and constructive feedback, and for recognizing our clear motivation, training-free design, and strong performance under aggressive compression. Below, we clarify the algorithmic contributions and distinctions from ForestPrune, and present additional experiments on practical efficiency, baseline comparability, and cross-architecture generalization.

W1&Q1. Algorithmic Novelty and Distinction from ForestPrune.

At a high level, DART follows a fundamentally different compression paradigm from ForestPrune and related token-compression methods. These methods typically start from fine-grained visual tokens and reduce their number through token merging, grouping, or pruning. ForestPrune, for example, organizes frame-level tokens into cross-frame token forests and then compresses them through node pruning.

DART instead performs top-down spatiotemporal splitting. It starts from a single cuboid covering the complete video feature volume and recursively splits this cuboid into smaller regions along the temporal axis (T), the spatial axes (XY), or both jointly (TXY). The process continues until the token budget is reached, and each final leaf cuboid is represented by one pooled token. Consequently, the output is a complete and non-overlapping partition of the original spatiotemporal feature grid.

This difference changes the basic budget-allocation problem. Merging- or pruning-based methods decide which existing tokens should be combined or removed. DART decides where additional resolution should be created by splitting the current regions. At each step, candidate splits are ranked according to their importance-weighted reconstruction-error reduction per additional token. The token budget is therefore allocated to the regions and axes that provide the largest reconstruction benefit, while temporal and spatial resolution can be increased independently.

Our novelty is not tree construction or greedy optimization in isolation. It lies in coupling top-down spatiotemporal splitting with reconstruction-guided budget allocation and axis selection. This formulation retains the spatial and temporal support of each final region in the partition and explicitly controls how the token budget is distributed across the video volume. Adaptive-T is a boundary-selection mechanism within this top-down framework: after a temporal split is selected, it chooses the temporal boundary that gives the largest reconstruction benefit. Thus, DART provides a resolution-allocation mechanism based on recursively refining a complete spatiotemporal volume, which is conceptually distinct from reducing a set of fine-grained tokens through merging or pruning.

AuthorsOpenReview ↗

Official Comment by Authors

W2&Q2. Practical Efficiency and Compression Overhead.

We measure latency under identical settings: LLaVA-OneVision-7B, one RTX 3090, 32 frames, FP16/SDPA, batch size 1, and greedy generation of exactly 32 tokens. We evaluate 100 fixed MVBench questions over five rounds, with 10 warm-up requests per method per round. Each request independently performs video decoding, visual encoding, compression, and generation. We report the median across five repeats per question, followed by the median across questions. All latencies are in milliseconds.

MethodBefore LLM Retained RatioVisual Prep Latency ↓Planning Latency ↓Prefilling Latency ↓TTFT ↓End-to-End Latency ↓End-to-End Speedup ↑Avg. Score ↑
Full100%618.7–1615.63055.13787.11.00×58.4
HoliTom25%681.0–390.81894.52581.21.47×58.8
EarlyTom25%496.7–391.11720.92407.91.57×58.2
FlashVID25%659.9–333.01810.52496.91.52×58.5
DART (Ours)25%740.1118.9422.31972.32657.11.43×59.1

DART’s compression costs are broken down below and included in the reported visual-preparation and end-to-end latencies.

ComponentLatency
Attention-based importance extraction, incremental overhead¹3.94 ms
Partition planning: feature transfer, normalization, prefix statistics, candidate search, and reconstruction-error evaluation118.9 ms
Pooling: native spatial downsampling and region pooling3.41 ms

¹ Attention-extraction overhead is measured using three paired extraction-on/off vision forwards per video on the same 100 videos, isolating the additional cost of obtaining importance maps from the encoder’s existing attention computation.

“–” denotes no separately isolated planning stage. TTFT spans video loading through first-token selection; end-to-end latency additionally includes the remaining generation and request finalization. Independently aggregated medians need not sum exactly.

At 25% retention, DART reduces prefilling latency, TTFT, and end-to-end latency by 73.9%, 35.4%, and 29.8% relative to Full, respectively, achieving a 1.43× end-to-end speedup with compression overhead included. It also achieves the highest four-benchmark average accuracy among the compared methods (59.1%, versus 58.4% for Full), demonstrating practical efficiency gains while preserving accuracy.

W3&Q3. Baseline Reliability and Evaluation Comparability

We additionally evaluate HoliTom, EarlyTom, and FlashVID under our unified LLaVA-OneVision-7B evaluation pipeline across all four benchmarks and retention ratios. The complete results are reported below. “Avg. Score” denotes the unweighted average across the four benchmarks, and “Relative” is normalized by the Full baseline’s average.

MethodPre-LLM retentionMVBench ↑EgoSchema ↑LongVideoBench ↑Video-MME ↑Avg. Score ↑Relative (%) ↑
Full100%58.360.456.458.658.4100
HoliTom25%58.061.056.458.958.6100.3
EarlyTom25%57.860.356.258.858.399.8
FlashVID25%58.060.257.658.358.5100.2
DART25%58.461.357.359.359.1101.2
HoliTom20%58.460.956.558.758.6100.3
EarlyTom20%57.660.455.158.157.899.0
FlashVID20%57.860.157.358.358.4100
DART20%57.961.457.359.359.0101
HoliTom15%58.160.855.457.758.099.3
EarlyTom15%57.160.454.557.557.498.3
FlashVID15%57.360.057.157.458.099.3
DART15%58.061.256.157.858.399.8
HoliTom10%57.761.255.656.957.999.1
EarlyTom10%55.859.453.155.055.895.5
FlashVID10%56.959.754.256.456.897.3
DART10%57.960.856.057.458.099.3

Under this unified evaluation, DART achieves the highest average accuracy at all four retention ratios. At 25% retention, it exceeds HoliTom, EarlyTom, and FlashVID by 0.5, 0.8, and 0.6 percentage points, respectively. These results show that DART’s average-performance advantage persists after reproducing the baselines within our pipeline, addressing the concern about cross-publication comparability.

AuthorsOpenReview ↗

Official Comment by Authors

W4&Q4. Generalization to Different Visual Architectures.

We implement DART on Qwen3-VL-2B-Instruct, retaining the reconstruction objective, budget-normalized action selection, and Adaptive-T. The architecture-specific adaptation consists of:

  • Grid and importance: use Qwen’s post-merger temporal–spatial feature grid and derive importance from Qwen vision self-attention, aggregated to that grid.
  • Feature integration: apply the same partition and pooling weights to the final visual features and aligned DeepStack features.
  • Sequence integration: update video-token placeholders, timestamp grouping, and positional indices to accommodate the compressed sequence.

The complete 32-frame results are shown below. “Avg. Score” is the unweighted four-benchmark average; “Relative” is computed from the displayed averages relative to Full.

MethodPre-LLM retentionMVBenchEgoSchemaLongVideoBenchVideo-MMEAvg. ScoreRelative (%)
Full100%61.853.953.357.456.6100
HoliTom25%60.552.851.055.855.097.2
EarlyTom25%60.252.750.454.554.596.3
FlashVID25%59.150.952.155.554.496.1
DART25%60.054.152.455.955.698.2
HoliTom20%60.452.550.254.754.596.3
EarlyTom20%59.051.850.154.153.895.1
FlashVID20%58.750.751.954.453.995.2
DART20%59.853.552.454.755.197.3
HoliTom15%58.951.849.454.153.694.7
EarlyTom15%56.950.649.453.152.592.8
FlashVID15%57.949.851.353.553.193.8
DART15%58.852.452.053.654.295.8
HoliTom10%57.249.948.952.452.192.0
EarlyTom10%56.248.748.851.051.290.5
FlashVID10%56.449.050.452.152.091.9
DART10%57.350.951.052.753.093.6

DART achieves the highest average accuracy at all four retention ratios, exceeding the strongest baseline by 0.6, 0.6, 0.6, and 0.9 percentage points, respectively. These results support transfer of its reconstruction-guided allocation mechanism beyond SigLIP and LLaVA-OneVision.

Reviewer_NmC5OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces DART, a training-free method for video token compression. It progressively partitions video features along temporal and spatial axes based on reconstruction gain per additional token, then pools each final region into one token.

Critical strengths
  • DART provides an interpretable token allocation strategy. A shared reconstruction criterion determines where to preserve temporal or spatial detail. This allows resolution to adapt to local video content.

  • The reported accuracy remains competitive under substantial compression. On the 7B model, average accuracy drops by only 1.1 percentage points when token retention decreases from 25% to 10%. Similar trends are reported on the 0.5B model.

Critical weaknesses
  • The reconstruction objective may overlook information needed for reasoning. Question-independent attention may underweight brief events or small details. Pooling regions without encoding their duration or extent may also obscure temporal relationships.

  • The added computational complexity is not sufficiently justified. Adaptive-T and joint splitting improve ablation accuracy by only 0.2 and 0.3 percentage points, respectively. Reported FLOPs exclude compression overhead and decoding costs, leaving the actual accuracy–latency trade-off unclear.

  • Generalization beyond the evaluated setting remains unclear. Experiments are limited to LLaVA-OneVision with 32-frame inputs. Evaluation on Qwen3-VL or a newer video-capable model, together with longer frame sequences, is needed.

Questions for authors

See Weakness Above

6 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the Reviewer for the thoughtful and constructive feedback. We clarify how DART preserves reasoning-relevant information, analyze the accuracy–latency trade-off of Adaptive-T and TXY, and further evaluate its generalization across visual architectures and input lengths.

W1. Reasoning-Relevant Information and Temporal Structure.

Question-independent importance. Our question-independent importance signal allows the partition and compressed visual tokens to be reused across multiple questions about the same video under the same sampling and token budget, amortizing the visual-compression cost. Under a fixed token budget, DART allocates tokens according to the attention-weighted reconstruction-error reduction per additional token. Since this criterion combines global importance with local feature variation, short events and fine-grained details that induce distinctive local changes can still receive additional temporal or spatiotemporal refinement. Adaptive-T further searches content-dependent temporal boundaries, allowing temporal resolution to be concentrated around informative temporal changes. Thus, the budget is preferentially assigned to regions that provide the largest information gain for downstream understanding.

At the task level, DART achieves 59.1% average accuracy at 25% token retention, compared with 58.4% for Full, while retaining only 25% of the original visual tokens. Under the more aggressive 10% retention setting, DART obtains 58.0% average accuracy compared with 58.4% for Full. These results show that DART preserves strong task performance under substantial token reduction, while its reconstruction-guided allocation continues to prioritize the visual regions most useful for downstream understanding.

Duration and spatial extent. These quantities participate in partition construction through each region’s spatiotemporal boundaries, although they are not explicitly appended to the output tokens. Pooling is restricted to individual contiguous cuboids. When representing temporally varying features with a single token incurs substantial reconstruction error, temporal or joint spatiotemporal refinement can allocate additional tokens to that region, subject to the shared token budget.

Token ordering and representation. The final tokens are ordered lexicographically by their temporal and spatial starting coordinates and receive the LLM’s ordinary consecutive positional indices. This preserves locality within each pooled region and a start-coordinate ordering across regions. The tokens are obtained by pooling the original projected features, keeping them in the embedding space expected by the frozen LLM.

AuthorsOpenReview ↗

Official Comment by Authors

W2. Computational Overhead and the Accuracy–Latency Trade-off.

We further evaluate the individual contributions of Adaptive-T and TXY under the same efficiency protocol at both 25% and 10% token retention. “DART w/o Adaptive-T” replaces the content-dependent temporal boundary with the midpoint boundary, while “DART w/o TXY” removes the joint spatiotemporal refinement operation. The results are shown below. All latency values include the corresponding planning and compression costs, and the reported speedup is relative to Full.

MethodPre-LLM retentionVisual prep latency (ms)↓Planning latency (ms)↓Prefilling latency (ms)↓TTFT (ms)↓E2E latency (ms)↓E2E speedup↑Avg. Score↑
Full100%618.7—1615.63055.13787.11.00×58.4
DART (Ours)25%740.1118.9422.31972.32657.11.43×59.1
DART w/o Adaptive-T25%721.0100.3421.91951.12639.81.43×58.6
DART w/o TXY25%734.8115.1421.41966.62650.71.43×58.4
DART (Ours)10%715.096.1187.11729.62404.91.57×58.0
DART w/o Adaptive-T10%696.580.5187.21711.12386.51.59×57.1
DART w/o TXY10%713.794.7187.11727.62403.21.57×57.2

Adaptive-T consistently improves accuracy at both retention levels. At 25% retention, it improves the average accuracy from 58.6% to 59.1%, while adding only 18.6 ms of planning latency and 17.3 ms of end-to-end latency. At 10% retention, its benefit becomes more pronounced: it improves the average accuracy from 57.1% to 58.0%, with an additional 15.6 ms of planning latency and 18.4 ms of end-to-end latency. This indicates that content-dependent temporal boundary selection is particularly useful when the token budget is more constrained, since it can allocate temporal resolution to the most informative parts of the video.

TXY provides a complementary benefit. Removing TXY decreases the average accuracy from 59.1% to 58.4% at 25% retention and from 58.0% to 57.2% at 10% retention. The corresponding planning overhead of retaining TXY is small: 3.8 ms at 25% retention and 1.4 ms at 10% retention. Thus, TXY helps preserve information that requires joint temporal and spatial refinement, while Adaptive-T focuses the available temporal resolution on content-dependent temporal changes.

Despite these additional components, DART remains substantially faster than Full. It achieves a 1.43× end-to-end speedup at 25% retention and a 1.57× speedup at 10% retention, reducing end-to-end latency by 29.8% and 36.5%, respectively. Overall, the ablation results show that Adaptive-T and TXY make complementary contributions: Adaptive-T provides content-dependent temporal allocation, while TXY preserves joint spatiotemporal details. Their accuracy gains are obtained with only modest additional latency, while DART continues to provide substantial end-to-end acceleration.

AuthorsOpenReview ↗

Official Comment by Authors

W3. Generalization Across Models and Input Lengths.

To assess generalization beyond the original LLaVA-OneVision setting, we evaluate DART on Qwen3-VL-2B-Instruct with both 32-frame and longer 48-frame inputs. In each table, “Avg. Score” denotes the unweighted average across the listed benchmarks, and “Relative (%)” is normalized by the corresponding Full-model average.

Qwen3-VL-2B-Instruct with 32-frame inputs. We evaluate DART and the compared methods on MVBench, EgoSchema, LongVideoBench, and VideoMME.

MethodPre-LLM RetentionMVBench ↑EgoSchema ↑LongVideoBench ↑VideoMME ↑Avg. Score ↑Relative (%) ↑
Full100%61.853.953.357.456.6100.0
HoliTom25%60.552.851.055.855.097.2
EarlyTom25%60.252.750.454.554.596.3
FlashVID25%59.150.952.155.554.496.1
DART (Ours)25%60.054.152.455.955.698.2
HoliTom20%60.452.550.254.754.596.3
EarlyTom20%59.051.850.154.153.895.1
FlashVID20%58.750.751.954.453.995.2
DART (Ours)20%59.853.552.454.755.197.3
HoliTom15%58.951.849.454.153.694.7
EarlyTom15%56.950.649.453.152.592.8
FlashVID15%57.949.851.353.553.193.8
DART (Ours)15%58.852.452.053.654.295.8
HoliTom10%57.249.948.952.452.192.0
EarlyTom10%56.248.748.851.051.290.5
FlashVID10%56.449.050.452.152.091.9
DART (Ours)10%57.350.951.052.753.093.6

DART achieves the highest average accuracy among the compared compression methods at all four tested retention ratios. Its average scores are 55.6, 55.1, 54.2, and 53.0 at 25%, 20%, 15%, and 10% retention, respectively. At 25% and 10% retention, it preserves 98.2% and 93.6% of the corresponding Full-model average accuracy.

Longer 48-frame inputs on the same model. We further evaluate all compared methods on Qwen3-VL-2B-Instruct with 48-frame inputs, using MVBench and VideoMME at 25% and 10% token retention.

MethodPre-LLM RetentionMVBench ↑VideoMME ↑Avg. Score ↑Relative (%) ↑
Full100%61.059.660.3100.0
HoliTom25%59.958.059.097.8
EarlyTom25%59.357.658.597.0
FlashVID25%58.757.558.196.4
DART (Ours)25%60.458.659.598.7
HoliTom10%57.754.356.092.9
EarlyTom10%57.553.655.692.2
FlashVID10%56.754.055.491.9
DART (Ours)10%58.255.456.894.2

DART achieves the highest average accuracy among the compared compression methods at both retention ratios. Its average scores are 59.5 and 56.8, exceeding the strongest baseline by 0.5 and 0.8 percentage points, respectively, while preserving 98.7% and 94.2% of the corresponding Full-model average accuracy. These results support DART’s effectiveness on Qwen3-VL-2B-Instruct under both tested input lengths.

Reviewer_DoU6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes DART, a training-free video token compression method. DART represents videos as partitions of spatiotemporal regions and employs a greedy algorithm to dynamically select temporal (T), spatial (XY), or joint spatiotemporal (TXY) refinements based on importance-weighted reconstruction error reduction. Experiments show that DART maintains or slightly surpasses the full-token baseline accuracy while retaining only 25% of the visual tokens.

Critical strengths

In my view, DART can also be considered a token pruning framework. However, unlike conventional methods that directly select or discard tokens based on their importance, DART incorporates importance into the reconstruction loss and guides spatiotemporal partitioning based on the reconstruction gain per additional token. I find this perspective interesting.

Critical weaknesses
  1. The overall presentation and writing need substantial improvement, as the current paper requires much effort to follow and understand.

  2. The greedy algorithm is central to DART, yet the paper provides no proof regarding optimality or, at least, approximate performance. It also lacks comparisons with alternative partition search strategies, making it difficult to assess whether the greedy approach is well justified.

Questions for authors
  1. DART repeatedly evaluates the reconstruction error of candidate partitions. What are the computational complexity and actual runtime overhead of this planning process? Since the paper only reports LLM prefill FLOPs, could the authors provide comparisons with other baselines in terms of actual prefill latency, Time-to-First-Token (TTFT), and end-to-end inference latency?

  2. Is the current combination of partitioning strategies optimal or empirically well justified? Why is Adaptive-T applied only to T splits, while the temporal boundaries in TXY splits remain fixed at the midpoint? Have the authors explored other combinations of partitioning operations or adaptive boundary selection strategies?

  3. Figure 4 is the only figure that provides examples of the resulting partitions. Could the authors include more visualizations across different video scenarios and token retention ratios to better demonstrate the adaptive behavior of DART?

3 comments
AuthorsOpenReview ↗

Official Comment by Authors

We thank the reviewer for the thoughtful assessment and constructive feedback, and for finding our reconstruction-guided perspective interesting. Below, we address the presentation concerns, clarify the planning complexity and practical efficiency, and provide additional experiments on partitioning design and greedy search.

Q1. Computational Complexity and Practical Inference Efficiency.

Planning complexity. Let S=HWS=HW denote the number of spatial visual tokens per frame, N=TSN=TS the total number of visual tokens, dd the feature dimension, and BB the retained-token budget. DART first constructs spatiotemporal prefix statistics in O(Nd)O(Nd) time. These statistics allow the reconstruction error of each candidate cuboid to be evaluated in O(d)O(d) time, independently of the cuboid’s volume. During refinement, the planner caches evaluated region scores and maintains a priority queue, generating candidates only for newly activated child regions. With O(B)O(B) activated regions and at most T−1T-1 temporal boundary candidates examined per region, the conservative complexity of prefix-based planning is

O(Nd+BTd+Blog⁡B).O(Nd+BTd+B\log B).

The first term corresponds to one-time prefix-statistic construction, the second to candidate boundary evaluation, and the third to priority-queue maintenance. For comparison, framewise pairwise matching or clustering in EarlyTom and HoliTom requires O(S2d)O(S^2d) comparisons per frame, or O(TS2d)O(TS^2d) over TT frames. FlashVID also involves dense pairwise comparisons in its compression modules. DART instead relies on prefix-cached reconstruction queries during region refinement and therefore avoids forming dense spatial-token similarity matrices in its search.

Practical efficiency. We evaluate all methods using LLaVA-OneVision-7B on a single RTX 3090, with 32 input frames, FP16/SDPA, batch size 1, and greedy generation of exactly 32 tokens. We use 100 fixed MVBench questions over five measured rounds, with 10 warm-up requests per method per round. Each request independently performs video decoding, visual encoding, compression, and generation. We report the median across five repeats per question, followed by the median across questions. All latencies are in milliseconds. Avg. Score is the four-benchmark average reported in the main paper.

MethodPre-LLM retentionVisual prep ↓Planning ↓Prefill ↓TTFT ↓E2E latency ↓E2E speedup ↑Avg. Score ↑
Full100%618.7—1615.63055.13787.11.00×58.4
HoliTom25%681.0—390.81894.52581.21.47×58.8
EarlyTom25%496.7—391.11720.92407.91.57×58.2
FlashVID25%659.9—333.01810.52496.91.52×58.5
DART (Ours)25%740.1118.9422.31972.32657.11.43×59.1

DART’s 118.9 ms planning time includes feature transfer, normalization, prefix-statistic construction, and partition search, and is already included in visual preparation, TTFT, and end-to-end latency. TTFT starts before video loading and ends at first-token selection; end-to-end latency additionally includes the remaining generation and request finalization.

At 25% token retention, DART achieves the highest average score of 59.1 among the compared methods, outperforming HoliTom, EarlyTom, and FlashVID by 0.3, 0.9, and 0.6 percentage points, respectively. Meanwhile, DART reduces prefilling latency, TTFT, and end-to-end latency by 73.9%, 35.4%, and 29.8% relative to Full, achieving a 1.43× end-to-end speedup with all compression costs included. These results demonstrate that DART combines strong task performance with practical inference acceleration, improving the average score over Full by 0.7 percentage points while retaining only 25% of the visual tokens.

AuthorsOpenReview ↗

Official Comment by Authors

W2&Q2. Partitioning Design and Greedy Search Justification

Partitioning operations and adaptive boundaries. We evaluate alternative operation combinations and boundary-selection strategies at 25% token retention. All planning latencies are in milliseconds.

ConfigurationTXYTXYMVBench ↑VideoMME ↑Planning Latency ↓
Full–––58.358.6–
DARTmidpointmidpointmidpoint58.258.8100.3
DART (Ours)Adaptive-Tmidpointmidpoint58.459.3118.9
DARTAdaptive-TmidpointAdaptive-T + midpoint-XY58.158.7150.6
DARTAdaptive-TAdaptive-XYmidpoint57.958.5104.2
DARTAdaptive-TAdaptive-XYAdaptive-T + midpoint-XY57.958.4120.3
DARTAdaptive-TAdaptive-XYAdaptive-TXY57.858.6402.2
DARTAdaptive-Tmidpoint–58.358.7123.4
DARTAdaptive-T–midpoint57.056.692.5

Our default configuration—Adaptive-T for T, with midpoint boundaries for XY and TXY—achieves the highest accuracy on both benchmarks among the tested configurations. Compared with all-midpoint splitting, it improves MVBench and VideoMME accuracy by 0.2 and 0.5 percentage points, respectively, while increasing planning latency from 100.3 to 118.9 ms.

Adaptive-T enables content-dependent temporal refinement through the T operation. Extending adaptive temporal boundaries to TXY does not improve accuracy in our experiments: it yields 58.1 on MVBench and 58.7 on VideoMME, with 150.6 ms planning latency. Adaptive-XY variants also yield lower accuracy, while the fully adaptive TXY configuration increases planning latency to 402.2 ms. Removing XY or TXY likewise reduces accuracy. These results empirically support applying adaptive boundaries to T only: content-dependent temporal boundaries improve temporal allocation at modest search cost, whereas extending adaptivity to XY or TXY increases planning cost without improving accuracy. Thus, midpoint boundaries provide a more favorable accuracy–efficiency trade-off for XY and TXY.

Greedy search versus Exact DP. To assess the quality–cost trade-off of the greedy planner, we compare its reconstruction distortion and runtime with Exact DP under the same partitioning objective at 25% token retention.

MethodPlanning Latency (ms) ↓End-to-End Latency (ms) ↓Mean Normalized Distortion Gap ↓Score
DART (Greedy)118.92657.10.274%61.0
Exact DP463.92980.20%60.0

The mean normalized distortion gap is computed by subtracting the Exact DP reconstruction distortion from the Greedy distortion for each video, normalizing by the distortion of the unsplit root region covering the entire video feature volume, and averaging the resulting percentages across 100 videos.

Greedy achieves 3.90× faster planning and 10.8% lower end-to-end latency than Exact DP, with a mean additional distortion of only 0.274% of the root-region distortion. It also achieves 61.0% question-answering accuracy, compared with 60.0% for Exact DP on this subset. These results support greedy search as an efficient planner that closely approximates the reconstruction optimum while preserving strong downstream task performance.

Q3. Additional Visualizations across Scenarios and Retention Ratios

We provide an additional figure in the supplementary material showing DART’s partitions across three video scenarios—human interaction, object motion, and local appearance—at all four token retention ratios: 25%, 20%, 15%, and 10%. For each video, we align the same sampled frames across budgets to facilitate direct comparison. Alongside the spatial partition overlays, we include temporal cross-sections through the same patch row, making the regions’ temporal extents directly visible. These visualizations illustrate how DART allocates spatial and temporal resolution across different video content and how regions span more frames on average as the token budget decreases. We further visualize an actual Adaptive-T refinement, showing the selected boundary and the reconstruction distortion at every candidate cut to clarify its content-dependent temporal boundary selection.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer DoU6,

We would like to briefly follow up on our response. To address your concerns, we have clarified the planning complexity and practical inference efficiency of DART, evaluated alternative partitioning and boundary-selection strategies, compared the greedy planner with Exact DP, and provided additional visualizations across different scenarios and retention ratios.

We would be grateful if, when convenient, you could take another look at our response and let us know whether these additions address your main concerns. We would be happy to clarify any remaining questions.

Thank you again for your time and consideration.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Contributions and Additional Evidence

Dear Area Chair,

Thank you for your time and constructive feedback. For ease of reference, we summarize the main contributions clarified and the additional evidence provided during the discussion.

1. A distinct perspective on visual token compression.

DART formulates compression as budgeted spatiotemporal resolution allocation. Starting from one region covering the visual feature volume, it progressively splits regions through temporal, spatial, or joint spatiotemporal refinements. Each refinement is evaluated by its attention-weighted reconstruction-error reduction per additional token. This top-down formulation jointly determines where to allocate tokens and which dimension to refine, providing a distinct design perspective relative to bottom-up token merging and token pruning.

2. Additional justification for the partitioning and search design.

We supplemented the boundary-design ablations with comparisons against exact dynamic programming under the same recursive refinement rules. These comparisons show small reconstruction gaps with substantially lower planning cost for greedy search. The boundary ablations further examine the choice of adaptive temporal boundaries while retaining midpoint spatial and joint boundaries. Together, these experiments provide empirical evidence for the adopted search and boundary choices.

3. Measured practical efficiency.

We supplemented complexity analysis with actual planning, prefill, time-to-first-token, and end-to-end latency measurements. In the reported LLaVA-OneVision-7B experiment on an RTX 3090 at 25% retention, DART achieves a 1.43× end-to-end speedup over Full, including compression overhead. Presenting these measurements alongside task scores makes the accuracy–efficiency trade-off explicit.

4. New evidence beyond multiple-choice evaluation.

To address the concern about random guessing, we added open-ended video captioning on 3,178 YouCook2 segments from 414 source videos, using LLaVA-OneVision-7B with 32 input frames. At a nominal 25% visual-token budget, DART achieves the highest BLEU-4 (0.92), METEOR (9.36), and CIDEr (25.84) among the compared compression methods, HoliTom, EarlyTom, FlashVID, and DART. Metrics are computed per source video and then macro-averaged, with scores multiplied by 100. These results extend the evaluation to free-form generation without answer candidates.

5. Evidence on an additional model architecture.

Additional evaluations on Qwen3-VL-2B-Instruct across four video-understanding benchmarks extend the empirical evaluation beyond the original LLaVA-OneVision backbone and examine DART's adaptation to a different visual-feature and positional interface.

We appreciate the opportunity to clarify these contributions and strengthen the empirical evidence. Thank you for considering our responses and additional experiments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.