Evi-Prefill: Evidence-Aware Prefill for Efficient Long Video Understanding

Existing acceleration methods for long video large models adopt a $\text{passive prefill}$ paradigm that uniformly processes video tokens. This ignores the sequential arrival of task-relevant $\text{visual evidence}$: once sufficient evidence emerges early, models still $\text{densely}$ process the full video stream, leading to redundant computation. To extend video temporal coverage, chunk-wise inference adopts partial observability yet remains $\text{dense}$. In this paper, we present $\text{Evi-Prefill}$, an $\text{evidence-aware active prefill}$ framework allocating subsequent compute from visual evidence accumulated under partial observability. Specifically, we formulate evidence gathering for incoming video chunks as a Partially Observable Markov Decision Process. With this model, we estimate evidence demand via the model’s internal signals and actively switch from $\text{dense to sparse}$ inference, reducing redundant compute while preserving performance. Moreover, Evi-Prefill filters structurally redundant frames before visual encoding to boost evidence estimation efficiency. Evi-Prefill requires no training and complements prior methods. Evaluations over 3 benchmarks achieve 1.5–1.9$\times$ speedup without average performance degradation, opening a new route to efficient long video understanding via active visual-evidence acquisition. Code will be released.