acceptodds
Under review as a conference paper at ICLR 2027

PreViC: Predictive Visual Compression for Video Vision-Language Models

Abstract

Video vision-language models (VLMs) turn even a short video into thousands of visual tokens, and those tokens dominate the prefill that precedes every answer. Cutting them before prefill, without retraining the model, is where compression pays off in deployment. Training-free methods for this stage mostly rank a token by how it looks on its own, using saliency, diversity, or similarity to neighboring frames. Such a score says nothing about how much a token adds to what has already been kept, so it keeps detail that nearby frames already explain and discards small changes that they do not. We propose PreViC (Predictive Visual Compression), which keeps a visual token exactly when a compact visual memory fails to predict it. PreViC is training-free and prompt-agnostic, and runs before the language model. A few temporally novel anchor frames form that memory, and every other token is ranked by how poorly its nearest anchors predict it. Over three checkpoints and five retain ratios, PreViC has the highest four-benchmark average accuracy of any method at 12 of the 15 settings. On LLaVA-OneVision 7B it holds the uncompressed model’s accuracy on 50% of the visual tokens, an operating point no baseline we measured reaches on either 7B model. Large video VLMs can therefore be served far more cheaply without giving up accuracy. More broadly, token compression is better posed not as ranking tokens by how important they look, but as asking what a retained visual memory cannot already predict.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Visual Token Compression, Efficient VLM

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_XjUCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies a training-free visual-token compression before LLM prefill. PreViC selects temporally novel anchor frames, reconstructs each non-anchor token using local ridge regression, and retains tokens with large residuals alongside an anchor allocation. Across three VLMs, e.g., Qwen2.5,PreViC showed better accuracy with faster efficiency, along with some interesting evidence that this helps multi-question QA better.

Critical strengths
  • Useful training-free temporal compression, likely useful for readers who want to try it out
  • Experiments on multiple VLMs and several benchmarks suggest better performance and efficiency
  • Careful paired accuracy analysis, sensitivity tests, and separate prefill and TTFT measurements show the wide usability of the method.
Critical weaknesses
  1. How generalizable are the hyperparameters and the method? A common issue with training-free methods (and this paper is no exception) is that they use varied hyperparameters. This suggests that there is room for hyperparameter tuning to get better performance on different benchmarks. I appreciate the authors discuss the sensitivity of hyperparameters in Table 15, but I am wondering whether we need different sets of hyperparameters on different benchmarks?

  2. Given that this is a training-free method, I highly encourage the authors to try this on more recent VLMs, which are believed to have better training on long videos, e.g., Qwen3, Gemma.

  3. The three parts of the methods (R1, R2, R3) seemingly form a heuristic pipeline of selecting tokens, but the mehanisms are still fragmented, e.g., some parts are based on novelty, some parts are based on the heuristic of hierarchical allocations, etc. These further weakens the generalization of the methods and how other people in the community would like to practice the methods. I recommend the authors better clarify how the R1, R2, R3 could be combined. I would also encourage the authors to explore a very lightweight learnable way to combine the three parts, if possible. But this is not required if the authors have limited rebuttal time.

Questions for authors

See above

Reviewer_ozWkOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies video token compression independent of Video LLMs. Unlike prior methods that rank tokens by saliency, diversity, or inter-frame similarity, PREVIC focuses on the incremental information each token contributes beyond what has already been retained. Specifically, it retains tokens that cannot be predicted from a compact visual memory, aiming to reduce redundancy while preserving novel information. Extensive experiments demonstrate that PREVIC improves VideoLLM performance across four video understanding benchmarks.

Critical strengths

The paper is well constructed and easy to follow.

The paper provides comprehensive experimental evaluations across four widely used video understanding benchmarks and two popular VideoLLM families.

Critical weaknesses
  1. Limited novelty discussion. As illustrated in Figure 1, the paper mainly compares PREVIC with methods that make token selection decisions independently within each frame. However, several prior works [1-3] have already explored cross-frame redundancy reduction. The paper should better clarify its novelty and advantages over these closely related approaches.
  2. Limited model evaluation. The evaluation could be extended to more recent VideoLLMs, such as Qwen3-VL-8B or Qwen3.5-9B, to further demonstrate the generalizability of PREVIC.
  3. Limitations of frame novelty estimation. In R1, frame novelty is estimated based only on the preceding frame, which may struggle with videos containing recurring scenes or repeated scene transitions, as seen in benchmarks such as Video-MME and LongVideoBench. Moreover, it remains unclear how well this strategy scales to long videos (e.g., over 30 minutes).
  4. Incomplete comparisons and redundant presentation. Why is DyCoke, which also performs pre-LLM token reduction, excluded from Figure 3? Additionally, Figure 3 and Table 2 appear to present overlapping information.

[1] Fu, Tianyu, et al. "Framefusion: Combining similarity and importance for video token reduction on large vision language models." 2025 IEEE/CVF International Conference on Computer Vision (ICCV), 2025. [2] Hyun, Jeongseok, et al. "Multi-granular spatio-temporal token merging for training-free acceleration of video llms." 2025 IEEE/CVF International Conference on Computer Vision (ICCV), 2025. [3] Shen, Xiaoqian, et al. "Longvu: Spatiotemporal adaptive compression for long video-language understanding." ICML, 2025.

Questions for authors

Please see weakness.

Reviewer_nPv4OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work addresses visual token compression for video LLMs. PreViC is a training-free method that operates before tokens enter the LLM. It first selects anchor frames, uses their tokens to reconstruct tokens from other frames, and prioritizes tokens with large reconstruction errors under a prescribed budget. Experiments on LLaVA-OneVision-0.5B, LLaVA-OneVision-7B, and Qwen2.5-VL-7B across four benchmarks demonstrate competitive performance compared with existing compression methods.

Critical strengths
  • The paper addresses an important problem: reducing the computational cost of video understanding while preserving model performance.
  • The method is training-free and operates entirely before the LLM, making it convenient to integrate into existing pipelines.
  • Evaluation across three models and four benchmarks provides useful evidence of its effectiveness.
Critical weaknesses

The novelty relative to ApET needs stronger justification. The core idea of reconstructing tokens from a reference basis and using reconstruction error to guide compression already appears in ApET. PreViC introduces differences in reference construction, cross-frame reconstruction, and budget allocation, but it remains unclear which of these changes is primarily responsible for the improvement. More direct, controlled comparisons would help establish the contribution.

Repeated content can carry essential task information. Low reconstruction error does not necessarily imply that a token can be safely removed. For example, in a video conveying Morse code through flashing lights, the timing and repetition of flashes are essential information, even when the corresponding visual features are nearly identical. PreViC may discard some of this evidence because it is predictable from the anchor pool. The moving-count performance drops in Table 13 are consistent with this concern, although they do not establish the underlying cause.

The fixed anchor pool may fail to remove genuinely redundant content. Several nearly identical frames may all have large reconstruction errors if their tokens are poorly represented by the anchor pool. These frames could therefore receive substantial token budgets despite contributing little additional information relative to one another. Because newly selected tokens are not added to the reference pool, the method does not explicitly account for redundancy among those selected tokens.

Questions for authors

See weaknesses above.

1 comment
Reviewer_nPv4OpenReview ↗

Response to Authors

Thanks for the authors’ clarifications and additional experiments. The controlled comparisons help clarify the relationship to ApET, although the main source of improvement remains insufficiently. The largest gain still combines several changes, and some skills show relatively small improvements. My concern about the clarity of the novel contribution therefore remains. W2 and W3 are also only partially addressed.

Nevertheless, this work has relatively comprehensive evaluation, competitive empirical performance, and practical value of a training-free, prompt-agnostic compression method. Overall, I consider the paper borderline, with a slight lean toward acceptance.

Reviewer_ufn3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Clear motivation and a practical method for video token compression. The idea of using predictive residuals to measure token redundancy is interesting, and the paper provides solid experiments across multiple VLMs and benchmarks.

My original major concern is the novelty compared with existing reconstruction-based compression methods, especially ApET. The authors have provided additional experiments in the rebuttal. Besides, I still have some concerns about the selection strategy of the pipeline (See weakness). Overall, I think the empirical contribution is solid enough for acceptance.

Critical strengths

Clear motivation and interesting insight. The paper proposes measuring visual token redundancy through predictive residuals rather than relying only on saliency or similarity. This provides an interesting perspective on video token compression.

Solid empirical validation. PreViC achieves strong performance across multiple VLMs, four video benchmarks, and different retention ratios. The authors also provide extensive baseline comparisons, ablations, and statistical analysis. The additional experiments in the rebuttal further strengthen the evaluation.

Practical and efficient design. PreViC is training-free, prompt-agnostic, and operates entirely before LLM prefill. It does not require modifications to the language model and achieves meaningful reductions in inference time and memory.

Critical weaknesses
  • Redundancy among selected tokens is not explicitly considered.

PreViC computes predictive residuals against a fixed anchor pool. However, two nearly identical non-anchor frames may both have large residuals if they are poorly represented by the anchors. The method may therefore retain redundant tokens from both frames.

The authors acknowledge this limitation in the rebuttal. I think it is worth further discussion, since reducing redundancy is the central motivation of the paper.

  • Limited evaluation of long visual sequences.

Although the authors provide additional results on videos longer than 30 minutes, the experiments still use at most 32 sampled frames. Therefore, the results do not fully demonstrate how the method scales to much longer visual token sequences. If your visual compression strategy is good, it should also improve long video understanding abilities[1][2], which would also strengthen your contributions.

[1] Wang, X., Si, Q., Zhu, S., Wu, J., Cao, L., & Nie, L. AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding. Findings of ACL 2025, pp. 5417–5432.

[2] Wan, Z., Wu, Z., Liu, C., Huang, J., Zhu, Z., Jin, P., Wang, L., & Yuan, L. LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference. Findings of EMNLP 2024, pp. 4065–4078.

Questions for authors

My major concern is Weakness 1. The additional comparisons with ApET are helpful, but I would like to better understand why the full PreViC pipeline is preferable to the simpler Anchor + Per-frame Budget variant.

Overall, I think the method is practical and the empirical results are convincing. The remaining issues could be further explored in future work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.