- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces ViCoS, a training-free method for video qa with limited visual input. ViCoS uses a component called VerCoRe, which allows the VLM to examine the currently retrieved frames. If the evidence is not enough, it will generate new concepts to guide new retrieval. Additionally, it uses a component called CoMo, which assembles unselected frames into a single image grid. The final input consists of high-quality frames plus this compact image. ViCoS is evaluated on two different backbones, five video qa benchmarks, and three compression budgets.
Critical strengths
-
ViCoS is a training-free method, and it can be applied directly to any VLMs. The whole test-time-scaling style pipeline is also clear and reasonable.
-
Table 2-3 in the paper distinguishes the specific contributions of two component and their combination to ViCoS. Table 4 compares various retrieval strategies. Table 5 shows the latency comparison between ViCoS and some closely related works. These ablation studies further show the paper's specific contributions.
Critical weaknesses
-
Although this paper claims ViCoS reduces the visual token budget, but it is only for the final input to VLMs. Actually, in the whole ViCoS pipeline, it processes more frames and visual tokens than the uniform baseline. For example, for K=2, based on Table 5 ViCoS uses 1.4 verifier calls, and it is 8.4 frame inputs. This is reflected in the latency (9.68s is double the uniform baseline of 4.88s, also is the slowest one). Also, the latency of uniform-K=8 (which as the baseline in Table 1) is missing now. This is important cause on many benchmarks the performance of uniform-K=8 is higher than or similar to ViCoS-K=2. Without the comparison on the latency, I am concerned if ViCoS actually provides a better accuracy-efficiency balance.
-
ViCoS contains two components (VerCoRe and CoMo), but based on Table 2, VerCoRe seems not to help alone. Adding VerCoRe alone even has some degradation at K=2. But combining VerCoRe and CoMo brings huge improvements. This is an interesting result but there is no detailed analysis of this interaction. It is important for me because if we combine the results on MVBench from Table 1-3, adding VerCoRe or CoMo alone only improves by 2 points, but combining them brings 14 points improvement. This is worth adding a detailed analysis. Otherwise, the contribution of each component is still not clear.
-
This point is related to the previous one. The idea of each component is already explored in VLMs. For example, VideoAgent [1] already used iterative retrieval with CLIP and LLM-based verification as VerCoRe. IG-VLM [2] and Video Panels [3] already used image grids to compact multiple frames into one image. Unless the authors provide a convincing explanation of what is new in combining these two components and why their combination can lead to much larger improvements than using either component alone, I do not think the novelty and contribution of this paper are enough.
-
Table 4 shows that verification only improves the performance by 0.31 and 0.09 over content-word retrieval on Qwen2.5-VL-7B on VideoMME, and the verification requires extra VLM calls. I am concerned if the additional VLM verification is really necessary.
[1] Wang et al. VideoAgent: Long-form Video Understanding with Large Language Model as Agent. ECCV 2024.
[2] Kim et al. An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM. arXiv 2024.
[3] Doorenbos et al. Video Panels for Long Video Understanding. CVPR 2026.
Questions for authors
-
Could you provide the end-to-end latency of Uniform-K=8 in Table 5 and compare it with ViCoS-K=2?
-
Could you explain why combining VerCoRe and CoMo brings much larger improvements than using any component alone, especially on MVBench?
-
Could you clarify the novelty of ViCoS compared with VideoAgent [1], IG-VLM [2], and Video Panels [3]?
-
Could you provide the same verification ablation as Table 4 on InternVL3-8B, especially on MVBench, to show how much the additional VLM verification contributes?