acceptodds
Withdrawn from ICLR 2027

StreamPrune: Causal and Query-Free Token Pruning for Accelerating Streaming Video MLLMs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal Large Language Models (MLLMs) have made significant progress in visual-language reasoning, including long- and short-term video understanding, but sampling video at high frame rates remains computationally expensive. Perceiving fast-changing or transient events requires dense frame sampling, yet the cost of processing dense visual tokens scales with frame rate. Moreover, consecutive frames in a video stream are often highly similar, causing much of this computation to be spent on redundant content. We propose StreamPrune, a training-free, query-free, plug-and-play token pruning module for streaming video LLMs that discards visually redundant tokens using only inter-frame similarity, measured by the Temporal Token Similarity (TTS) module. It is independent of the downstream query and transfers across multiple streaming MLLM backbones. We further introduce the Content-Adaptive Caching (CAC) module, which retains only temporally novel tokens and organizes them in a content-adaptive cache that is compatible with existing streaming retrieval. Across six streaming video benchmarks, StreamPrune reduces visual tokens by 92.6 ± 1.8% and increases encoding throughput by 6.18×, at an average accuracy drop of only 0.8 ± 2.9 points (both spreads are standard deviations across the six benchmarks), while outperforming existing pruning methods on StreamingBench and StreamBench. Code will be released publicly after the review process.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video LLMs, Streaming Video Understanding, Efficient Inference, Streaming Inference, Token Pruning, KV Cache Compression

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oTMmOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes StreamPrune, a training-free and query-free token pruning module for streaming video MLLMs. TTS keeps a token only when its cosine distance to the last kept token at the same position exceeds τ. CAC then packs the kept tokens across frames into 196-token blocks. With ReKV-7B at 1 FPS, the paper reports a 92.6% token reduction, 6.18× higher throughput and an average accuracy drop of 0.8 points over six benchmarks at τ=0.70. The authors claim that it outperforms existing pruning methods on StreamingBench and StreamBench.

Critical strengths
  1. The method is causal, query-free and training-free, which fits the streaming setting. It can be added to ReKV without changing any model weights, so it is easy to deploy.
  2. At τ=0.50, the KV-cache growth drops from 37.7 to 10.6 GB/h, while the average accuracy is the same as the unpruned model. This memory saving is the main practical benefit.
Critical weaknesses
  1. All three abstract numbers use τ=0.70, but StreamBench accuracy at this τ is 5.3 points below unpruned ReKV in Table 4a. In addition, three Avg values in Tables 7 and 8 differ from their sub-task means while other rows match. ReKV should be 64.4 and 46.7, not 63.0 and 45.7, and StreamPrune should be 45.1, not 45.4. These three errors all favor StreamPrune. Both problems are the main reason for my soundness score.
  2. The core change of TTS over RLT brings little gain. In Table 4a, prev is the rule of RLT, and ref differs from it by only −0.6 to +1.4 points for τ≤0.70. The ref rule clearly leads only at τ≥0.80, where both rules are below 60. This limits the novelty of the paper.
Questions for authors
  1. Table 2a has no pruning baseline, and Tables 2 and 3b only compare methods at τ≤0.60. Can RLT and TimeChat-Online be added as baselines on both benchmarks, and can all methods be compared at τ=0.70?
  2. CAC adds 1.5 points in Table 4b. Do the baselines in Table 2b also use CAC and the same token budget?
  3. Which τ is used in Tables 7 and 8, and why is unpruned ReKV 70.3 on StreamingBench in Table 3a but 69.1 elsewhere?
  4. What causes the drops of about 10 points on STU and HLD in Tables 7 and 8?
  5. Although the introduction uses denser frame sampling as the motivation, all experiments use 1 FPS. It would be better to compare StreamPrune at 2 to 4 FPS with unpruned ReKV at 1 FPS under matched throughput.
Reviewer_TwKGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

StreamPrune is a training-free, query-free token-pruning method for causal streaming video processing. TTS maintains the most recently retained feature at each spatial location and retains a new token only when its cosine distance from that reference exceeds a threshold. This avoids cumulative drift when comparisons are made only against the preceding frame. CAC packs surviving tokens from different frames into fixed-size cache blocks, translating sparsity into reduced language-model prefill and KV-cache growth. The main evaluation uses ReKV-7B with 1 FPS input. At threshold 0.70, the paper reports 92.6% mean visual-token reduction, encoding throughput increasing from 11 to 68 frames/s, cache growth decreasing from 37.7 to 3.6 GB/hour, and a 0.8-point mean accuracy drop across six benchmarks. The reported ±1.8% and ±2.9 are standard deviations across benchmarks, not confidence intervals from repeated trials. Additional experiments cover other backbones, comparison references, and cache layouts.

Critical strengths

The reference-update rule is simple and directly addresses streaming redundancy. Updating each spatial reference only when a token is retained prevents a sequence of drops from leaving the current token unconstrained relative to the stored reference. This aligns the comparison with actual retained content more directly than comparing with the preceding frame. In Table 4(a), ref improves over prev by 1.0, 5.3, and 4.3 points at thresholds 0.70, 0.80, and 0.90, respectively, consistent with greater benefit during long runs of discarded tokens. The rule uses only current and past features and fits streaming scenarios in which future queries are unknown.

Combining pruning with cache organization produces tangible systems benefits. CAC addresses the failure of per-frame padding to realize memory savings after pruning. With the same surviving tokens, Table 4(b) improves StreamBench accuracy from 63.6 for TTS alone to 65.1 for TTS+CAC, showing that layout and effective retrieval capacity matter. The main table reports token reduction, encoding throughput, and cache growth rather than compression alone. Threshold 0.50 also provides a less aggressive operating point that preserves the reported mean accuracy while increasing throughput from 11 to 37 frames/s.

Critical weaknesses

Comparisons with the closest compressors lack budget-alignment details, and averages conceal substantial variation. Table 2(b) and Appendix Table 6 report accuracy for ToMe, VisionZip, VidCom2, and STC without jointly reporting retained-token ratios, throughput, KV size, or full configurations under a common retrieval budget. The 5.6-point advantage over STC-Cacher & Prune therefore does not yet establish superiority at matched resources. The StreamChat throughput comparisons in Table 3(b) are useful, but do not replace budget matching among compressors on the same backbone. Aggressive compression also transfers unevenly: at threshold 0.70, StreamBench falls from 65.8 to 60.5, and Flash-VStream falls from 48.8 to 41.2 on StreamingBench. A small average cost should be distinguished from preserved quality on every task or backbone.

A key cache-boundary behavior is unspecified, and the high-input-frame-rate motivation lacks direct validation. Section 2.2 and Figure 3 describe prefill after a 196-token block is completed, but do not specify how an incomplete tail buffer is handled when a question arrives. In a mostly static scene, a few important changed tokens may take a long time to complete a block; whether they are flushed immediately directly affects information availability and response latency. The main experiments use 1 FPS input. Encoding throughput of 68 frames/s does not directly establish real-time performance or transient-event accuracy at higher sampling rates with frequent queries.

Questions for authors

Could accuracy–throughput–KV-size curves be reported for StreamPrune, STC-Pruner, and STC-Cacher & Prune using the same ReKV backbone and retrieval budget, together with realized token-retention ratios for every configuration?

When a question arrives before the CAC tail block is full, are the available tokens immediately forwarded to the LLM? If so, is flushing included in throughput and query-latency measurements? Please explain timestamp/position handling after cross-frame packing and validate content freshness and latency under at least one higher-input-FPS, frequent-query setting.

Could per-benchmark accuracy and token reduction be provided for each threshold on the 7B model, explaining how the OVO and VStream sub-settings are combined and how the six-benchmark means and standard deviations are computed? Please also reconcile the StreamingBench baselines of 70.3 and 69.1. This would help verify the headline averages and their task-level variation.

Reviewer_8saWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes StreamPrune, a causal, query-free, and training-free visual-token pruning method for streaming video MLLMs. For each spatial position, Temporal Token Similarity (TTS) compares the current visual token with the most recently retained token and discards it when the cosine distance does not exceed a threshold. Content-Adaptive Caching (CAC) then packs the surviving tokens from multiple frames into fixed-size cache blocks. The method is evaluated on six streaming-video benchmarks and several backbones. At the highlighted operating point, the paper reports a 92.6% token reduction, a 6.18x increase in encoding throughput, and an average accuracy decrease of 0.8 points.

Critical strengths
  1. The problem setting is practically relevant. Reducing the continuously growing visual-token and KV-cache cost of streaming video models is important. The causal and query-free constraints are appropriate for applications in which questions are not known when frames arrive.

  2. The method is easy to integrate into existing streaming systems. The last-retained anchor has a clear intuitive motivation and gives an explicit distance bound relative to the cached representation.

  3. The paper evaluates six benchmark families and demonstrates transfer to two additional backbones on StreamingBench. The reported efficiency improvements are substantial: at τ=0.70, encoding throughput increases from 11 to 68 frames/s, while the reported KV-cache growth rate decreases from 37.7 to 3.6 GB/h.

  4. The paper is generally clear and easy to follow. Figure 4 provides an intuitive visualization of where TTS retains tokens. The limitations section also discusses camera motion, irreversible pruning, small-object changes, and the need to choose τ in advance relatively candidly.

Critical weaknesses
  1. Missing closely related work and baseline. TimeChat-Online (ACM MM 2025) introduces a closely related causal and query-free Differential Token Drop method that removes temporally redundant visual tokens while preserving their spatial-temporal positions. It reports the same central observation that more than 80% of streaming-video tokens can be removed without language guidance, yet it is neither cited nor compared. Eventful Transformers (ICCV 2023) also compares incoming tokens with references stored at their last update, making it an important conceptual precedent for the proposed last-retained anchor.

  2. The high-frame-rate motivation is not tested. The paper argues that StreamPrune enables dense sampling needed for transient events, but all accuracy experiments use 1 FPS. It does not show that a pruned model at 2/4/8 FPS can outperform an unpruned 1-FPS model under the same computation or memory budget.

  3. CAC’s online behavior is underspecified. It is unclear what happens when a query arrives while the current 196-token block is only partially filled. If the block is not processed, the answer cannot use the most recent evidence; if it is flushed, the resulting query latency should be measured. The paper also does not explain sufficiently how timestamps, frame boundaries, and spatial-temporal positions are preserved after tokens from multiple frames are packed together. The throughput protocol lacks details such as stream duration, repetitions, CPU/host-memory configuration, and whether query-time flushing and data transfers are included.

  4. Unclear operating-point selection. Different claims use different thresholds, and no selection protocol or validation set is described. The headline token reduction, throughput, and accuracy change use τ=0.70, but the comparisons use τ=0.50 (StreamingBench) and τ=0.60 (StreamBench). At τ=0.70, StreamBench accuracy is 60.5 (Table 4(a), ref), 5.3 points below the unpruned model and below all StreamChat configurations. The abstract also claims gains over pruning methods on StreamBench, although Table 2(a) contains no pruning baseline.

  5. The comparisons are not conducted at matched efficiency budgets. Table 2(b) and Tables 6–8 omit the baselines' token-retention rates, throughput, and memory, so the accuracy gaps may reflect different budgets rather than better pruning criteria. Similarly, Table 4(a) compares frame, prev, and ref at the same τ, but different reference rules need not yield the same retention rate: accumulated drift can make ref keep tokens that prev discards, so its advantage at τ greater than or equal to 0.60 may partly reflect a larger budget.

Questions for authors
  1. How does StreamPrune differ technically and empirically from TimeChat-Online’s Differential Token Drop? Can the authors provide a matched-budget comparison if implementation permits? In addition, please clarify the conceptual relationship between TTS and the last-update reference gating used in Eventful Transformers.

  2. Can the authors evaluate StreamPrune at actual input rates above 1 FPS? In particular, does a pruned 2/4/8-FPS model outperform an unpruned 1-FPS model at matched computation or memory on tasks involving brief or fast-changing events?

  3. How were the thresholds used in the different tables and benchmarks selected? Was a validation set used? At τ=0.70, tokens with cosine similarity of at least 0.30 to the cached reference are discarded; can the authors provide similarity distributions or semantic/event-level analyses that calibrate this threshold?

  4. What are the token-retention rates, throughput, and cache-memory costs of the baselines in Table 2(b) and Tables 6–8? Can the authors also report accuracy–retention or accuracy–throughput curves for the three strategies in Table 4(a), and determine whether ref remains better than prev at matched retention rates?

  5. What happens when a query arrives before the current CAC block is full? Is the partial block immediately prefilled, and is this latency included in the efficiency measurements? How are timestamps, frame boundaries, and spatial-temporal positions preserved after cross-frame packing?

  6. Could the authors clarify why the unpruned StreamingBench score is 70.3 in Table 3(a) but 69.1 in Tables 1(b), 2(b), and 6; how the two OVO-Bench splits and the two VStream-QA settings aggregated into the six benchmark-level values used to compute the reported mean and standard deviation; and which value of τ is used in Tables 7–8?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.