- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors propose VideoRouter, a training-free framework based on temporal hierarchical representation in videos and the MLLM-based self-verification mechanism. VideoRouter first constructs a global answer and a local answer, and then leverages a voting router to decide the final response (or fallback if the requirement is not met). The experiments demonstrate the effectiveness of the framework across different MLLMs, and it also outperforms previous video agents. In the ablation studies, the authors also discuss the impact of temporal hierarchy and the routing mechanism.
Critical strengths
- The authors use the global-local hierarchical representation in long-form videos, which grasps the challenge that the downstream tasks in such videos do either relate to overall comprehension or local fine-grained perception/reasoning.
- The routing mechanism looks reasonable: first assess the soundness of the initial prediction, and then merge them into the final response. This design makes the model adaptively take advantage of the evidence distribution in the video sequence based on the task itself (global or local).
- The ablation discusses the main modules carefully (hierarchy and routing).
- The effectiveness of the VideoRouter is demonstrated in the main experiments.
Critical weaknesses
-
The evaluation setting is not clear. What is the base model performance? In table 1, only Local and Global is provided, but on videomme, the performance for qwen2.5-VL-7B is 65.1(w/o subs)/71.6(w/ subs) from the its tech report; and on LongVideoBench validation set the performance is 56.0. In this case, the improvement on Videomme is only around 0.4 and on longvideobench is as high as 8.23.
-
The inference cost. The authors provide this in the appendix D. And compared with AKS, VideoRouter is more efficient in token usage. However, what about compared with the base model itself, e.g., Qwen2.5-VL-7B? It is also of great value to see the trade of between performance and efficiency when equipped with such a strategy. If, as mentioned in weakness 1, the improvemtn is limited (e.g., in videomme), the extra computation cost will be viewed as useless.
-
How to demonstrate the effectiveness in the verification process since the same model is used here? How many correct prediction can score 2 and how many wrong prediction can score 1 or 0?
-
After fallback, is there any further improvement? Only fallback rate is provided in the appendix.
-
Is there any evidence to illustrate that the two branches indeed capture the global story and local details?
Questions for authors
See above in Critical Weakness.