- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Evi-Prefill, a training-free pipeline for efficient long-video understanding. It combines SSIM-based frame filtering with attention-guided switching from dense to sparse chunk processing to reduce visual encoding and LLM prefill costs. Experiments on two LLaVA-based backbones and three video benchmarks report additional speedups over existing compression methods while maintaining or slightly improving average accuracy.
Critical strengths
- Practical inference efficiency. The method reduces both visual encoding and LLM prefill costs through frame filtering and adaptive chunk processing. The reported speedups, with comparable or slightly improved average accuracy, provide a meaningful practical benefit for long-video inference.
- Training-free compatibility. The pipeline can complement existing compression methods without additional training. Results across multiple compression methods and two backbones support its usefulness as an add-on optimization.
Critical weaknesses
- Limited methodological novelty. The method appears to be a well-engineered combination of existing heuristics rather than a substantial methodological advance.
- Insufficient conceptual insight. The paper does not convincingly establish that its attention-based heuristic measures evidence sufficiency rather than serving as a pipeline-specific scheduling rule.
- Limited backbone coverage. Evaluation uses only two LLaVA-based backbones, leaving applicability to newer architectures, such as Qwen3-VL and InternVL, unclear.
- Limited benchmark coverage. Broader evaluation on long-video(LVBench, InfiniBench) and egocentric benchmarks(EgoLifeQA) is needed to substantiate the method’s generality.
- Insufficient fine-grained analysis. Aggregate results do not establish robustness across video categories, evidence locations, and tasks requiring temporally distributed reasoning.
Limited methodological novelty remains my main concern, so addressing the empirical gaps alone may not be sufficient for me to raise my score.
Questions for authors
- Novelty: What is the substantive methodological contribution beyond combining existing frame filtering, attention thresholding, and sparse processing heuristics?
- Evidence sufficiency: How do you distinguish declining NCM caused by its cumulative normalization from a genuine reduction in the need for additional evidence?
- Matched-budget comparison: Does attention-guided scheduling outperform fixed or uniform sampling at the same latency or processed-frame budget?
- Generalizability: Do you have evidence that the gains extend to newer backbones and more challenging long-video or egocentric benchmarks?
- Failure cases: How does performance vary across video categories, particularly for late-occurring evidence and temporally distributed reasoning?
15 comments
Response to W2
W2: The paper does not convincingly establish that its attention-based heuristic measures evidence sufficiency rather than serving as a pipeline-specific scheduling rule.
We appreciate the opportunity to clarify the insights behind our method.
We identify an observation in streaming processing: New Chunk Attention Reveals Evidence Demand (see Figure 2). This phenomenon holds not only for early evidence but also for later evidence (see Figure 7). We leverage this signal as a proxy for proactive evidence acquisition under partial observability (see Section 4.1). Our approach shows promising performance across different methods and datasets (see Table1-2), and extends effectively to additional backbone architectures and datasets (see following response). These results suggest that our approach goes beyond pipeline-specific scheduling. We hope our modeling of streaming evidence acquisition and insights into proactive prefill will offer a new perspective on efficient long video understanding.
Response to Q2
Q2: How do you distinguish declining NCM caused by its cumulative normalization from a genuine reduction in the need for additional evidence?
Thank you for this question. The key distinction is between position-dependent decay and evidence-dependent variation. Reviewer q8Ak raised a similar concern. Following the suggestion of q8Ak, we test whether NCM responds to evidence when processing position and prefix history are held fixed.
On LVBench, we construct paired inputs with the same question, identical prefix, and identical chunk position, replacing only the current chunk with either an annotated evidence chunk or a visually matched chunk from the same video outside the annotated evidence interval. The visual-token budgets and timestamp prompts are also matched.
Let denote the shared cumulative prefix mass. Then:
Therefore:
With the prefix and position controlled, this difference isolates the current chunk’s attention response rather than a change in processing position.
| Chunk position | Mean NCM: Evidence (×10²) | Mean NCM: Control (×10²) | Relative increase |
|---|---|---|---|
| 4.31 | 3.73 | +15.4% | |
| 2.29 | 1.99 | +14.6% |
At matched chunk positions, evidence chunks exhibit 15.4% and 14.6% higher mean NCM than their controls. Their mean unnormalized attention mass is also higher by 2.60 and 2.26, respectively, indicating that the separation is already present before cumulative normalization. These results support that NCM captures evidence-sensitive information beyond the normalization-induced positional trend. Accordingly, Section 4.1 uses its recent trajectory as an observable surrogate.
Response to W1&Q1
Thank you for your constructive comments. It is worth noting that Evi-Prefill is a simple yet effective extension acceleration method. However, simplicity does not imply limited novelty. Our contribution extends efficient long video understand from compressing acquired visual representations to allocating subsequent computation based on observed evidence. We clarify this contribution in four aspects.
1. Identifying passive prefill redundancy that remains after compression. The evaluated compression methods reduce per-chunk processing costs but still densely traverse the input video. We instead consider the sequential arrival of task-relevant evidence and its uneven contribution across time. The following table (from Tables 1–2) shows that redundant computation in existing methods can be reduced while maintaining or improving performance, demonstrating the complementarity of visual compression and temporal evidence allocation for further acceleration.
2. Establishing an empirical link between structural redundancy and internal attention. Figures 2 and 9 show overlap between structurally similar frames and low-attention regions, motivating filtering before visual encoding. Frame Allocation (FA) uses structural similarity, requiring neither an external model nor pre-extracted visual features. We validate this design through evidence retention and matched-budget comparisons. On the 1,415 LVBench questions whose reference intervals contain candidate frames, FA thus maintains high annotated-evidence coverage while reducing the input frames.:
We further match uniform sampling to FA’s retained frame count per question, with the host compressor fixed and EA disabled. At identical frame budgets, FA improves overall accuracy over uniform sampling, with gains of 1.3/1.6 points on long videos, supporting its selection value:
3. Turning internal observations into a concrete temporal evidence allocation policy. We formulate subsequent evidence acquisition as sequential decision-making under partial observability, using question-conditioned new-chunk attention as a lightweight proxy signal. To assess this policy, we fix FA and the host compressor and match uniform sampling to EA’s actual processed-chunk count per question. At the same processing budget and comparable latency, EA improves overall accuracy by 0.4/0.7 points, including 1.4/1.3 points on long videos. This supports the value of observation-guided allocation beyond simply processing fewer chunks:
4. Providing a perspective complementary to representation compression (the most crucial novelty of this paper). These designs establish two transferable insights:
Evi-Prefill provides a training-free implementation and experimental baseline for this direction, compatible with existing token compressors. We will highlight the innovations of this paper in subsequent updates to the manuscript.